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Chan’s book explores the challenges in assessing experiential learning, deepens 
our understanding and inspires readers to think critically about the purpose of 
assessment in experiential learning. 

Experiential learning has been studied and proven to be effective for student 
learning, particularly for the development of holistic competencies (i.e. 21st cen- 
tury skills, soft skills, transferable skills) considered essential for individuals to 
succeed in the increasingly global and technology-infused 21st century society. 
Universities around the world are now actively organising experiential learning 
activities or programmes for students to gain enriching and diversified learning 
experiences; however, the assessment of these programmes tends to be limited, 
unclear and contested. 

Assessment plays a central role in education policies and students’ approach 
to learning. But do educators know how to assess less traditional learning such 
as service learning, entrepreneurship, cross-discipline or cross-cultural projects, 
internships and student exchanges? While the current assessment landscape is 
replete with assessments that measure knowledge of core content areas such as 
mathematics, law, languages, science and social studies, there is a lack of as- 
sessments and research that focus on holistic competencies. How do we as- 
sess students’ ability to think critically, problem solve, adapt, self-manage and 
collaborate? 

Central to the discussion in this book is the reason students are assessed and 
how they should be assessed to bring out their best learning outcomes. Offering 
a collection of best assessment practice employed by teachers around the world, 
this volume brings together both theoretical and empirical research that under- 
pins assessment; and perceptions of different stakeholders — understanding of as- 
sessment in experiential learning from students, teachers and policymakers. The 
idea of assessment literacy also plays an important role in experiential learning, 
for example, reflection is often used in assessing students in experiential learning 
but how reflection literate are educators, are they aware of the ethical dilemmas 
that arise in assessing students? These questions are discussed in detail. The 
volume also introduces a quality assurance programme to recognise student de- 
velopment within experiential learning programmes. 


The book will be particularly informative to academic developers, teachers, 
students and community partners who struggle with the development and as- 
sessment for experiential learning, those who plan to apply for funding in expe- 
riential learning, and policymakers and senior managements secking evidence 
and advice on fine-tuning curricular, assessment designs and quality assurance. 
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Foreword 


Universities used to assess what was taught and what was easy to test, and in 
some they still do. This led to a focus on relatively low-level knowledge tested at 
the end of courses. This encouraged students to delay study until close to their 
examination and prompted them to adopt strategies of memorisation. However, 
they have gradually recognised that this promotes a very restricted curriculum 
which does not reflect the full range of learning outcomes that are needed to 
produce graduates who can adapt to the demands of a complex world. 

Most disciplines, especially those aligned to professions, have broadened their 
assessment strategies and few now rely solely on final examinations. The focus 
has moved to broadening what students learn. For example, the soft skills of 
communication and working cooperatively with others have taken on greater 
importance. Much of this change cannot be accommodated in traditional 
lecture-based courses. New learning modes and activities have been established 
appropriate for the outcomes being sought. Hands-on active approaches to 
teaching and learning have been adopted. Some of these involve students in 
real-world situations outside the educational institution in business, industry or 
the community. Students learn in the settings where work happens in response 
to the needs they meet there. Other active approaches utilise various forms of 
simulation and group work on campus. A great range of terminologies, often 
discipline-specific, are used to describe these activities — placements, practicums, 
work-integrated learning, teamwork, practice-sessions, etc. — which all fall under 
the general heading of experiential learning. Here, the term is used generically as 
a shorthand to encompass many different experience-based tasks and activities, 
on campus and beyond. 

What all these approaches have in common is that students learn by work- 
ing with experience. Students undertake tasks that focus on the capabilities and 
attributes they need to develop to be recognised as a graduate in a particular 
field. These range from well-equipped simulations of health environments where 
students practice being nurses or clinicians of various kinds, to simulations of 
engineering processes or stock markets, etc. Students are placed in either real 
or simulated contexts in which they have to appraise the situation they find 
there, intervene appropriately and make judgements about the effectiveness of 
what they do. They learn through acting and reflecting on what they do. Most 
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importantly, they learn in consort with others. Technical knowledge and skills 
combine with interpersonal attributes to address meaningful issues. Tasks are of- 
ten not individual but require collaboration with others, be they fellow students 
or others they may encounter in a particular context. 

Assessing the learning outcomes of such work requires us to think beyond 
conventional tests and exams, although there are places where technical knowl- 
edge of such practices is still useful. The range and scope of what needs to be 
judged are far greater than individual disciplinary knowledge or what can be 
judged through simple written assignments. Graduate attributes and transferable 
skills now take greater prominence. 

As the extent of such experiential activities has increased and as the formal 
establishing of course learning outcomes has acknowledged their importance, 
demands on assessment have never been greater. It is not a straightforward mat- 
ter of applying what we already know about test construction and the design of 
unseen examinations, but confronting major new challenges: How can hard-to- 
assess outcomes be evaluated? What new ways of thinking are required if what 
is to be learned does not reside within a given course unit? How can assessment 
be designed in ways that do not undermine the very kinds of learning that are 
being promoted? 

Cecilia Chan has been grappling with these issues with success for many years. 
In this book, she has brought together key ideas from the literature and from her 
own research and the practice of herself and her colleagues. This is a substantial 
undertaking as this area is fraught with diverse and conflicting terminology, a 
lack of consensus on key ideas and a dearth of good quality research. Her book 
helps the reader navigate the pathways of assessment and experience-based learn- 
ing. It identifies key ideas and dilemmas: for example, how to choose appropriate 
assessment practices, how to assess group work, how to involve external parties 
and how to resolve tensions between reflection and assessment? It is a valua- 
ble sourcebook which brings together practices from different disciplines which 
hitherto have not been discussed in one place. It provides ways of thinking about 
how generic attributes and discipline-specific learning can be assessed within 
one course. 

This book provides a stimulus for discussion on a challenging topic. It enables 
us to take more thoughtful approaches to the assessment of an increasingly im- 
portant aspect of higher education. 


David Boud 
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Prologue 


This book is written especially for teachers, academic developers, policymakers, 
students, activity organisers and community partners. I hoped to write a book 
with which readers can easily reflect, reinvent and reapply onto their own as- 
sessment practice for experiential learning. I didn’t want to write a book with 
only practical examples and not supported by research theories and frameworks. 
However, coming from a professional discipline background, I also understand 
the frustration of reading (and not understanding) a book with only educational 
research and theories. But as someone who has been leading academic develop- 
ment at a research-intensive university for over 15 years and conducting serious 
educational research, I recognise how important it is that the effective strategies 
we engaged are backed by theories and research evidence. This book bridges that 
gap. I have brought in my perspectives and experience of what I believe is most 
useful for readers in assessing experiential learning. I have first-hand experience 
in most of the assessment approaches I presented in the book, and truly practice 
what I preach. 

In the book, I tried to use simple daily examples to illustrate the different as- 
pects of experiential learning. Knowing the readers of this book may be teachers, 
educational developers, policy writers from various disciplinary backgrounds and 
types of post-secondary education institutions (such as community, vocational 
and technical colleges, research-intensive and teaching-intensive universities), I 
made it a point to use straightforward language and terms that are not specific 
to educational research, and I hope that I have managed to make it readable and 
approachable for all readers. The way this is written is not only for teachers to 
learn new tricks or improve old ones but also to allow policymakers and funders 
to fully understand and appreciate the work that teachers and administrative 
staff deal with in experiential learning, raising caution not to underestimate the 
workload and hurdles through which teachers and supporting staff have to jump. 

The examples, case studies and focus of the book are written for post-secondary 
education; however, most of the content is suitable for different levels of educa- 
tion. School teachers and principals can adapt some of the assessment strategies 
to suit the culture of their students and environments. 

There is one area that I did not cover, and it is on the assessment of experi- 
ential learning during the COVID-19 pandemic. During the pandemic, many 


xx Prologue 


experiential learning activities came to a halt, and some activities were reinvented. 
With some countries imposing travel restrictions, overseas experiential learning 
opportunities have also reduced significantly, leading to the introduction of the 
term experiential learning at home. Our Pro-Vice-Chancellor at the University 
of Hong Kong, Prof. Ian Holliday made a very sound point that there are many 
experiential opportunities to gain at home, and he encouraged teachers to take 
this opportunity to reinvent. Teachers have begun to use simulation or online 
flipped learning to replace the part of face-to-face. For example, my colleagues 
from Architecture, Mathew Pryor and Gavin Coates used concept boards, online 
studios and online field trips to help students to learn about landscape architec- 
ture. Tanya Kempston from the Faculty of Education guided university student 
mentors to work virtually with secondary school students to create radio dramas 
and broadcasted them at her virtual radio drama festival called “Hear This!” The 
Global Citizenship@Home internships at the Faculty of Social Sciences headed 
by Elsa Lam created a new pathway for students to achieve global learning with 
internship placements in Hong Kong. Rebecca Lam from the School of Chinese, 
Faculty of Arts used the YOCLE online platform (see Chapter 9) and asked 
students to identify Chinese phrases or words that are written incorrectly in 
restaurants, shopping malls or anywhere that they may encounter in their every- 
day lives. Students also have the opportunity to comment and like each post as 
well as messaging the class, bridging the learning isolation from the pandemic. 
Among these new reinvented experiential learning activities, many new techno- 
logical tools were trialled and employed. Yet, the actual assessment challenges 
and approaches in experiential learning remained relatively similar. The main 
differences were the result of using more technology tools during the pandemic. 
Thus, those challenges that fall in the category of using technology for assessing 
experiential learning, such as digital exclusion and breach of academic integrity, 
persisted. Of course, I am aware that there may be a lot of disagreement and 
debates regarding some of these so-called newly invented “experiential learn- 
ing,” where the experience may not be truly authentic, and the kind of student 
learning experience is reasonably doubted. Future research in this area is needed. 

In this book, I have used the generic term “Experiential Learning” to describe 
the range of activities that are experience-based and hands-on. Using this term 
as an umbrella term is by no means underplaying other equally important terms 
used to describe experience-based learning such as experience-based education 
or work-integrated learning. The term experiential learning is used as it is the 
most commonly known term to describe these activities in Hong Kong and 
many parts of Asia. As this book focuses on assessment, I didn’t want to distract 
readers’ attention with the ambiguities arising from the conceptualisations of 
the experiential learning terminologies. Thus, in Chapter 1 of the book, I have 
provided several concepts of experiential learning from researchers, teachers and 
students to uncloud the idea behind experiential learning. 

Each chapter begins with a quote that relates to the theme of the chapter, and 
the structure of the main body may differ depending on the theme of each chap- 
ter. The chapter ends with a section called questions to ponder and I included my 
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own personal reflection after completing each chapter which diverges from the 
conventions of a typical academic book. This book is written for academic devel- 
opers, teachers, policymakers, students, activity organisers and community part- 
ners more than it is for educational researchers. However, individual researcher 
will find useful examples and practices from this book and may contemplate the 
kind of findings that are needed to support these practices to help generate the- 
ories and frameworks. The questions and reflections at the end of each chapter 
may also inspire research questions. Below is a short summary of each chapter. 


Short Summary of Each Chapter 
Chapter 1: What IS and IS Not Experiential Learning? 


From a scholarly definition of experiential learning to a practical interpretation 
of it from the lenses of educators and students around the world, this chapter 
offers a tangible definition and typology of experiential learning in relation to its 
stakeholders. It also includes a list of benefits and challenges for different parties 
involved in experiential learning, which lay the foundation for a deeper dive into 
the topics in later chapters. 


Chapter 2: Experiential Learning Theories and Frameworks 


To prepare readers for a thorough discussion of the nuances that exist in experi- 
ential learning, this chapter offers an introduction and digestible breakdown to 
helpful theories and frameworks that address the fundamentals of the educational 
phenomenon. It also touches on holistic competency models, which are highly 
helpful in understanding the non-academic learning outcomes of learning experi- 
ences. From experience-related research like Dale’s Cone of Experience and Kolb’s 
Experiential Learning Theory, to holistic competency-oriented models such as Ho- 
listic Competency Development Framework (HCDF) and Chain of Mirrors, this 
chapter takes readers on an analytical journey covering the rationales, strengths, 
criticisms and applied examples of theories relevant to experiential learning. 


Chapter 3: Assessing Academic Knowledge and Experiential 
Learning 


There are numerous similarities and differences between assessing academic 
knowledge and holistic competencies. Deeply understanding the former gives 
us insights into the basics of assessment, including concepts like formative and 
summative assessment, assessment literacy and students as partners, which are 
highly useful for and applicable to assessment in experiential learning. This chap- 
ter connects the dots between assessments of these two very different groups of 
learning outcomes, and showcases the benefits and challenges brought about by 
the different types, methods and approaches to assessment in experiential learn- 
ing, all well-supported by recent educational research and examples. 
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Chapter 4: Designing Experiential Learning Assessment 


Following the big-picture discussions in the previous chapters, this chapter serves 
as a detailed guide to assessment design in experiential learning. The chapter be- 
gins with a list of essential questions for educators to ask oneself when deciding 
on assessment types and when designing the actual tasks in their programmes. 
Enriched with real-life examples and thinking prompts, I present nine common 
assessment tasks in experiential learning and their respective strengths, weak- 
nesses, effective design tips and sample rubrics. This is followed by eight expe- 
riential learning activities, from capstone, internships, to residential education, 
and concrete ways in which their learning outcomes can be assessed. 


Chapter 5: Reflection as Assessment in Experiential Learning 


This dedicated chapter digs into reflection as a common but tricky assessment 
for experiential learning, particularly as evidence for higher-order thinking pro- 
cesses before, during and after an experiential learning activity. Drawing from 
prominent interdisciplinary studies on reflection as a practice, this chapter ex- 
plores this complex topic and breaks it down from the perspectives of students, 
teachers and institutions, before expanding into its sociocultural nuances and 
reflection literacy, the understanding of which is the key to authentic and mean- 
ingful reflection in the context of education. The chapter also offers important 
principles for designing impactful reflective assessments and lays the foundation 
for a later discussion on ethics. 


Chapter 6: Feedback in Experiential Learning 


Feedback can be a challenging aspect in experiential learning, especially when the 
learning outcomes are personal and often unpredictable, making it difficult to 
standardise and maintain quality and usefulness of the feedback. In this chapter, 
I invite readers to examine factors affecting feedback: quality, sources of feedback 
(i.e., types of assessors), modes or practices, and feedback literacy, and prompt 
readers to critically evaluate the types of feedback given in various contexts. It 
also discusses the importance of evaluative judgement and how it can contribute 
to the development of feedback practices and literacy with further research. 


Chapter 7: Ethics in Assessing Experiential Learning 


Previous chapters have implied a range of ethical concerns surrounding expe- 
riential learning, its pedagogy and assessment, all of which are addressed with 
examples in this thought-provoking chapter. This chapter covers the concerns 
and dilemmas that educators should consider during design and other decision- 
making processes of experiential learning, and illustrates with possible scenarios 
for readers to ponder. This is followed by some suggested solutions to overcom- 
ing these challenges and tips for ensuring a safe space for students, teachers and 
other stakeholders and participants of experiential learning activities. 
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Chapter 8: Assessment Cases around the World 


This chapter provides an anthology of real-life experiential learning assessment 
examples from institutions around the world. These authentic and detailed 
case studies range from community service, field trips, internships, to hands-on 
practicum programmes, and were sourced from top universities from around 
the globe, including Hong Kong, Canada, the United States, the United King- 
dom, Singapore, Australia and New Zealand. In each case study, I guide readers 
through its learning activities, assessment methods, rubrics and other grading 
criteria, feedback from teachers and students, and more, making the chapter a 
highly useful reference resource for educators and programme designers. 


Chapter 9: Assessing Experiential Learning with Technology 


Technological advancements inspire innovations in every field, and education is 
not an exception. This chapter showcases how technology can be effectively uti- 
lised to enhance students’ experiential learning process either as a main or sup- 
port tool. I also remind readers of the challenges that may come with the misuse 
or inappropriate adoption of technology in certain situations and contexts. The 
chapter is wrapped up by a case study of an overseas construction safety online 
course using YOCLE, a multi-platform e-learning tool with built-in assessment 
tools and validated instruments designed specifically for experiential learning 
and holistic competency. 


Chapter 10: Quality Assurance and Evaluation in Experiential 
Learning 


Quality assurance and evaluation of learning and impact are both crucial to a 
programme’s success and sustainability, and potentially messy and intimidating. 
This chapter draws from my experience as a teaching and learning expert in 
higher education and a wide range of educational research to guide readers — 
especially potential grant applicants, administrators and policymakers in the field 
of experiential learning — through the evaluation process that is most useful and 
important for experiential learning. This includes critical guiding questions and 
evaluation methods and tools, as well as the International Holistic Competency 
Foundation (IHCF) Quality Assurance programme for programmes targeting 
holistic competency outcomes. 
Enjoy your reading, hopefully you will find the book useful. 


Yours sincerely, 
Cecilia Ka Yuk Chan 
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1 What IS and IS Not 
Experiential Learning? 


A valuable experiential learning curriculum allows students to reflect, relearn, 
react, reinvent, reform and reapply their learning. 
— Chan, CKY 


Introduction 


The wide-ranging definitions and ever-enhancing approaches in experiential 
learning, exacerbated by a lack of unified theories and frameworks, often cloud 
teachers as to what defines and constitutes the term “experiential learning.” 
For reasons of consistency, the term “experiential learning” is adopted in this 
book as a general, descriptive term to refer to a broad range of activities that 
centre on the intentioned process of learning where “experience [is consti- 
tuted] as a form of knowledge” (Usher, 1993, p. 169). Using this term as the 
generic term is by no means to underplay other equally important terms such as 
experience-based education or work-integrated learning. The term experiential 
learning is used as it is the most commonly known term to describe these activ- 
ities in Hong Kong and many parts of Asia. As this book focuses on assessment, 
I didn’t want to turn the readers’ attention to the ambiguities that arise from 
the conceptualisations of the experiential learning terminologies. Thus, this 
first chapter begins by putting together how educational scholars, teachers and 
students understand and experience experiential learning. This may help teach- 
ers and students uncloud the concept of experiential learning when they create, 
build and plan their own activities and focus the design on the more vital areas 
of “experiencing,” “reflecting,” “clasping, 


> “< > < > < 


applying” and “feedforwarding.” 


1.1 The Interpretation and Practice of Experiential 
Learning from Various Educational Researchers, 
Scholars and Associations 


Below are some direct quotes from educational researchers, scholars and asso- 
ciations who have been investigating various branches of experiential learning. 
These direct quotes serve as a guide for teachers and a quick literature review for 
emergent scholars in this area. 


DOI: 10.4324/9781003018391-1 


What IS and IS Not Experiential Learning? 


In the words of Lewis and Williams (1994, p. 5): “In its simplest form, 
experiential learning means learning from experience or learning by doing. 
Experiential education first immerses learners in an experience and then 
encourages reflection about the experience to develop new skills, new atti- 
tudes, or new ways of thinking.” 


During an interview with David Kolb (2012), he said: “Experiential learn- 
ing puts learning in the center of the learning process for a learner.” In his 
book (Kolb, 1984), he commented that “Experiential learning is the process 
whereby knowledge is created through the transformation of experience”. 
He interprets his ideas using his four-stage experiential learning theory, 
namely, concrete experience (by experiencing from prior or from new ex- 
perience), reflective observation (by reflecting on the experience), abstract 
conceptualization (by rethinking and clasping the experience), and active 
experimentation (by reapplying and testing what you learnt from the expe- 
rience and reflection). 


Another term that is often used synonymously is “experience-based learn- 
ing” (e.g., Andresen et al., 2000; Kraft, 1990; Moore, 2010). Andresen, 
Boud and Cohen (2000, p. 225) proclaimed “the distinguishing feature 
of experience-based learning (or experiential learning) is that the experi- 
ence of the learner occupies central place in all considerations of teaching 
and learning. This experience may comprise earlier events in the life of the 
learner, current life events, or those arising from the learner’s participa- 
tion in activities implemented by teachers and facilitators. A key element 
of experience-based learning (henceforth referred to as EBL) is that learn- 
ers analyse their experience by reflecting, evaluating and reconstructing it 
(sometimes individually, sometimes collectively, sometimes both) in order 
to draw meaning from it in the light of prior experience. This review of their 
experience may lead to further action.” 


Katula and Threnhauser (1999, p. 240) labeled experiential learning as 
a “process that takes place beyond the traditional classroom and that en- 
hances the personal and intellectual growth of the student. Such education 
can occur in a wide variety of settings, but it usually takes on a learn-by- 
doing aspect that engages the student directly in the subject, work or service 
involved.” 


Work integrated learning is another term that is used widely to describe 
work related experience, it is “the intersection and engagement of the- 
oretical and practice learning. The process of bringing together formal 
learning and productive work, or theory and practice. Constructing one 
system using available knowledge from several separate sources” (Cooper 
et al., 2010, p. xiii). 


The National Society for Experiential Education (2013) included a list of 


eight principles for experiential learning activities: “Intention”, “Preparedness 
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and Planning”, “Authenticity”, “Reflection”, “Orientation and Training”, 
“Monitoring and Continuous Improvement”, “Assessment and Evaluation” 
and “Acknowledgment”. 


1.2 The Interpretation and Practice of Experiential 


Learning from Teachers around the World 


Many teachers have shared their understandings and experience of what expe- 
riential learning is in programme documents and course promotional material. 
In order to better apprehend teachers’ perceptions and appreciate their efforts 
in designing experiential learning activities, a list of quotes has been selectively 
collected from teachers through course information leaflets and videos around 
the world. By demonstrating a wide variety of approaches and experiences, this 
aims to shed some light on how teachers interpret experiential learning. 


“Nowadays, Hong Kong society and teenagers do not lack knowledge and 
information. What we lack is the wisdom in applying knowledge, which is 
important for solving problems in different settings. Experiential learning 
provides the perfect opportunity for students in this regard.” Prof. Sam- 
son Tse (Dean of Student Affairs/Former Associate Dean, Faculty of Social 
Sciences, University of Hong Kong) 


“At the end of the term when they present their projects to the class, the 
clients and other professors, I have the privilege of witnessing how much 
my students have matured. They have incredible pride and increased confi- 
dence in their work.” “Experiential learning and the opportunity to see my 
students thrive are priceless.” Prof. Meral Demirbag Büyükkurt (Professor 
in the Department of Supply Chain and Business Technology Management, 
The University of Concordia) 


Anthony from the School of Public Health and Health Systems at the Uni- 
versity of Waterloo takes her students out of the classroom so that they 
can have face-to-face conversations with Duff, a man who lived on the 
streets for more than three years. She also brings them to local shelters, 
and even to federal prisons. “It’s intense and personal. But I want stu- 
dents to know that it’s reality, it’s their world,”; “It’s also their place 
in the world and their responsibility as citizens to engage in the com- 
munity,”; “I want my class to change the way students think about the 
world and about health. For better or worse, or better and worse, I want 
them to see the world, and their responsibilities in it, more clearly.” Ms. 
Kelly Anthony (Lecturer, School of Public Health and Health Systems, 
University of Waterloo) 


“Experiential learning is intense and often messy, but that messiness helps 
students understand and master the concepts that they’re learning in the 
lecture hall.” Dr. Kate Trimble (Senior Associate Dean and Director, Office 
of Experiential Learning, Massachusetts Institute of Technology, MIT) 
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“Only so much can be taught, the rest must be learned through experi- 
ence.” Dr. Adam Barsky (Edward Brown Award for Teaching Excellence 
recipient, Associate Professor, Faculty of Business and Economics, The Uni- 
versity of Melbourne) 


1.8 The Interpretation and Experience of Experiential 
Learning from Students around the World 


» 


With students being the *end-users," their perceptions of these experiential 
learning activities are probably the most important. It has long been evidenced 
in educational research that students’ attitudes influence their motivation to en- 
gage in learning (Gardner, 1988). Prosser and Trigwell (1999) proclaimed that 
how students perceive their learning can determine the effectiveness of their ac- 
ademic learning. Their *Student Approaches to Learning" model proposes that 
students’ perception of the learning environment will influence their adaptation 
of a particular approach to learning (i.e. deep or surface approaches to learning), 
which in turn mediates the learning outcomes achieved. 

Learning outcomes of experiential learning often focus on holistic competen- 
cies,! and scholars have been vocal about how students’ perceived importance of 
holistic competency development could affect their motivation to enhance these 
competencies (Chan & Luk, 2020; Chan, Zhao, & Luk, 2017). Students with 
more positive attitudes (e.g., enjoyment) demonstrate a higher level of engagement 
in experiential activities that lead to holistic competency development (Chan & 
Yeung, 2020). Thus, in the design of experiential learning activities, including as- 
sessment and feedback, it is vital to ensure a clear understanding and consideration 
of students’ perceptions. Any mismatch or large discrepancies in their understand- 
ings, expectations and actual experience may hinder the occurrence of learning 
and competency development. In light of that, a list of quotes has been collected 
from students around the world and compiled below to help readers conceive 
experiential learning through the lens of students’ unique angles and experiences. 


“To me, experiential learning is to touch upon the real world, and to learn 
about what is said to be possible in the textbooks but rarely implemented 
perfectly in the real world.” And “I think my experiential learning experi- 
ence was indeed a practical one, because I could apply what I have learnt 
from Public Administration, such as the collaborative dynamics between 
the government, the private sector and the third sector into my experiential 
learning experience." Karen Chung (Student, Department of Public Ad- 
ministration, Faculty of Social Sciences, The University of Hong Kong) 


“T have never felt more useful at university than my experience in the 
UNSW Human Rights Clinic. As someone not suited to conventional, 
classroom-style learning, the Human Rights Clinic allowed me to truly 
learn what it takes to start as a human rights lawyer, work in a team, and 
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learn from outstanding supervisors. I found the work to be extremely chal- 
lenging, but always felt set up for success by the support provided by my 
peers and supervisors. Working specifically in migration law not only gave 
me a thorough understanding on a human rights issue that is always in the 
news, but improved my drafting and legal analysis. In all honesty, I think an 
experience like the HRC should be a compulsory part of a law degree.” Sam 
Koslowski (Student, 2018, Faculty of Law, University of New South Wales) 


“T want to live in a community where people care about each other. If I 
want that, then I have to be that. Having an opportunity where I can get 
guidance from my professors and learn from mistakes before starting an 
articling position gives me comfort — when I go into articling, I'll have the 
necessary experience to do a better job in an area that I wouldn't have had 
practical work experience in." (3rd year student, Osgoode Hall Law School, 
York University, cited in Council of Ontario Universities, 2014) 


“T think experiential learning is interactive, it is practical and also it is very 
professional, so that I can get a more holistic education because it not only 
enforces my understanding of the theories, but it also improves my core 
values and my generic skills that can come in handy in my career.” Molly Gu 
(Student, The University of Hong Kong) 


*Over my one-semester abroad, I got the chance to make friends with people 
from all over the world. [...] What's more, as a university student, I experienced 
the different teaching styles at Queen's University and The Chinese University 
of Hong Kong. I do believe that I will become more proactive, considerate, and 
creative from now on, thanks to my experience as an exchange student." (Stu- 
dent, Faculty of Science, Chinese University of Hong Kong) 


“In fall of 2018, I went into Sustainability 350 not knowing what to expect. 
All I knew was that I would be required to volunteer at Ginkgo Organic 
Garden. What started out as ‘extra work’ quickly turned into an experience 
that changed my life. I learned about all the different areas of sustainability 
while committing myself to helping our environment. Lifelong friends were 
made, and I still volunteer on my own time because it makes me feel good 
to know I can make a difference. Help because you want to, not because you 
are being asked to. You will feel the change.” (Student, Human Resource 
Management programme, Roosevelt University, 2021) 


1.4 Criticisms on Experiential Learning 


There are a lot of ambiguities in the conceptualisations of the term *experiential 
learning," particularly from the researchers! perspectives. Researchers have ar- 
gued that the conceptualisations of experiential learning are not robust enough 
to support the variety of factors that occur in learning by experience. Below I 
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present some criticisms from researchers so that readers can receive an unbiased 
picture. Experiential learning 


e tends to focus on retrospective reflection, fails to address the “here and 
now” experience (Vince, 1998); 

° fails to account for social, political-cultural and institutional influences on 
the learning process (Reynolds, 1999); 

° places too much emphasis on the role of the individual learner and decon- 
textualises the learning process (Holman et al., 1997); 

e does not take into consideration the role of a learner's intention and desire 
in learning. The focus on cognitive reflection is simplistic (Britzman, 1998); 

e does not account for the interaction between cognition and the environment, 
how each individual's cognition converges with others,’ or “how individual 
knowledge co-emerges with collective knowledge" (Fenwick, 2000, p. 263). 


Researchers who are interested in deepening their knowledge on the precise 
conceptualisation of the terms used for experiential learning and the criticisms 
that arise from them should consult the articles above. 


1.5 Types of Experiential Learning 


Experiential learning can be divided into two major categories — those occurring 
outside of the classroom known as field-based experiences, and those occur- 
ring inside the classroom, often during class time, known as classroom-based 
learning. 

Field-Based Learning was first found integrated into higher education in the 
1930s. Field-based learning includes experiences such as internships, clinical ex- 
periences, rescarch and practical fellowships, apprenticeships, student exchanges, 
undergraduate research, practicums, cooperative education, service learning and 
community-based learning. Activities that bring together academic learning and 
practical application of theory such as internships, practicums, service learning 
and clinical experiences are also known as work integrated learning or work- 
based learning. While work integrated learning is increasingly becoming a dis- 
tinct field of research and practice, it is fundamentally based on the principles of 
learning through reflection, making sense of experiences and situated learning 
(Cooper et al., 2010). Hence, work integrated or work-based learning activities 
are included as part of field-based experiential learning in this book. 

Classroom-Based Learning can also be considered as experiential learning 
according to Lewis and Williams (1994). Classroom-based experiential learning 
includes role-playing, games, case studies, simulations, presentations, debates, 
discussions, hands-on technology and various types of group work. However, 
with the evolving nature of pedagogies, these types of experiential learning are 
now considered more as pedagogies for active learning. 

In this book, I will mainly focus on the assessment of field-based experiential 
learning activities, although some assessment practices with a little bit ofimagination 
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can also be used for classroom-based learning activities. Sometimes, there is no pre- 
cise boundary in categorising the types of experiential learning activities. 


1.6 Benefits of Experiential Learning to Different 
Stakeholders 


Depending on whose perspectives you are adopting, there will be different 
benefits associated with experiential learning. There now exist ample funding 
opportunities for universities, communities, corporations, teachers and even 
students to propose diverse learning experiences. Shell Canada, an oil and gas 
company, initiated the Shell Canada’s Campus Ambassador Programme that 
offers funding for students in many Canadian universities to provide oppor- 
tunities for their hands-on experience (Dalhousie University, 2015; University 
of Alberta, 2021; University of British Columbia, 2014; University of Cal- 
gary, 2016). The packages may include cash rewards and subsidies for travelling 
and activities organised by clubs and societies, with the priorities given to ones 
with objectives to improve students’ educational experience in relevant areas 
of Shell’s business (e.g., engineering, energy, sustainable energy, environment 
and economy issues), as well as opportunities to engage with the community 
at large. At the University of Hong Kong, experiential learning is a graduation 
requirement, and undergraduate students must engage in one overseas and one 
Mainland China learning experience to broaden their horizons and promote 
internationalisation (The University of Hong Kong, 2021). Support and funding 
resources for faculties and students to experience “educationally rich” real-life 
problem-solving activities (Gallant Ho Experiential Learning Centre, 2021) are 
available. The University of Hong Kong also provides teaching development 
grants for teachers to design projects and activities that will have an impact on 
the strategic development and promotion of teaching and learning; and experi- 
ential learning is marked as a priority area and a distinctive feature of the Univer- 
sity’s undergraduate curriculum. Residential colleges are often allocated funding 
resources for experiential learning, enabling residents to organise programmes 
that benefit competency development. At the University of Macau, a full res- 
idential collegiate system similar to that of Oxford, Cambridge and Yale has 
been implemented. Ample experiential learning activities are organised within 
the residential colleges to build competencies such as leadership, citizenship with 
global outlooks and teamwork (University of Macau, 2021). 

With abundant schemes funded by governments, corporations and universi- 
ties, grant proposers often require the applicants to identify benefits associated 
with their proposed projects. In this section, I provide a comprehensive list of 
possible benefits for different stakeholders in their proposals, using appropriate 
teaching and learning terminologies. 


Benefits for Universities (Boose, 2004; Faculty of Health University of York, 
n.d.; Fenton & Gallant, 2016; Ngai, 2006; Panet al., 2018; University of New 
Brunswick, 2014) 
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Increase student recruitment and retention by deepening student experiences; 
Enhance reputation among students in the network of employers; 

Promote university’s image as socially responsible, in particular when stu- 
dents are involved in community and service learning; 

Maintain network with alumni through partnership with their companies; 

Increase graduate employment as students are better prepared for work, and 
may even secure employment opportunities during internships, fellowships 
and services, etc.; 

Secure partnership opportunities and financial support from community or 
corporate partners; 

Publicise their pedagogical innovation which may lead to further govern- 
ment funding; 

Develop international network; 

Enhance university reputation by promoting diverse active authentic 
pedagogies. 


Benefits for Teachers (Boose, 2004; Cantor, 1995; Faculty of Health Univer- 
sity of York, n.d.; Fenton & Gallant, 2016; Ngai, 2006; Pan et al., 2018; Univer- 
sity of New Brunswick, 2014) 


Reignite teacher’s love of teaching as they provide meaningful education to 
students; 

Learn new approaches to interact with students; 

Apply new pedagogical approaches to students who have non-traditional 
learning styles; 

Guide students to achieve a deeper understanding of course concepts and 
their applications; 

Enhance and maintain student engagement, motivation and participation, 
which in turn lead to teacher fulfilment; 

Obtain rich data, research insights and connections which are possibly use- 
ful for future research; 

Enhance teacher satisfaction as experiential learning often gives teachers 
greater satisfaction than traditional teaching approaches; 

Provide opportunities for teachers to meet with potential research industrial 
partners; 

Senior management often values teachers who have industrial and commu- 
nity network; 

Often attract funding opportunities and even teaching award recognition 
due to the innovativeness and workload of experiential learning. 


Benefits for Students (Boose, 2004; Faculty of Health University of York, n.d.; 
Fenton and Gallant, 2016; Ngai, 2006; Pan et al., 2018; University of New 
Brunswick, 2014) 


Enhance academic motivation and engagement through bridging the gap 
between academic knowledge and the authentic world, hence encouraging 
self-directed learning; 
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e Increase clarity about academic goals and career expectations; 

e Appreciate other cultures and people from different backgrounds; 

e Develop professional networks and gain potential employment opportunities; 

e Gain real-world experiences, obtain information on selected career fields 
and build network; 

e Enhance holistic competencies (including cognitive competencies, values 
and attitudes) as well as specific skills (technical and professional skills); 

e Develop self-confidence and maturity; 

° Improve academic performance. 


Studies have shown modest or no differences between students with experiential 
learning experiences and those without (Strage, 2000). Although it is often re- 
flected that experiential learning students outperform non-experiential learning 
students in the same course (Freeman et al., 2014), it is unfair to say all types of 
experiential learning benefit students in terms of their academic performances. 
Moreover, even if experiential learning improves students’ grades, it is unlikely 
due to the enhanced mastery of course knowledge, but a deeper understand- 
ing and a better application of academic knowledge in the real world (Eyler & 
Halteman, 1981, as cited in Strage, 2000) and an improved attitude towards 
learning. 


1.7 Primary Purposes of Experiential Learning — 
Enhancing Student Learning 


Experiential learning brings together student learning opportunities that are 
otherwise difficult to provide in a normal classroom setting. The primary bene- 
fits are that it enhances student learning in areas of “relevancy”, “holistic com- 
petency development,” “reflection,” “meaningful feedback,” “value mistakes” 
and “actual experience”. 
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Relevancy — students often criticise the university curriculum for being irrele- 
vant to the real-world for example, “too many theories, not enough practice,” 
“too conceptual,” “teachers do not know what is going on in the real-world,” 
“my professor is still teaching me something that was used twenty years ago” — 
these criticisms can be found in many student experience evaluation reports in 
universities. Teachers are sometimes bounded by policies of the accreditation 
bodies or universities and may find it difficult to introduce a curriculum which 
is more “authentic” and “relevant.” Experiential learning differs in that it allows 
students to take what they have learnt and apply them first-hand. The learning 
processes, either good or bad, are more meaningful for them as they experience 
them and develop their own relevance. 


Holistic Competency Development - In a classroom, students are often given 
known solutions to problems, but in life, most problems are ill-defined and some 
may not even have solutions. Experiential learning trains students to be creative, 
to think critically, to work in teams (in most of our lifetime, we work in teams: 
as parents, as siblings and as teammates), to solve problems, to find ways to deal 
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with oneself and be resilient if the problems do not have solutions. Thus, holistic 
competencies are not just for students’ careers, but more importantly, for their 
personal life. Competency development is much more feasible in experiential 
learning activities for students to make meaning out of it. 


Guide by Reflection — experiential learning encourages spontaneous and in- 
tentional reflection that is not necessarily by design. For example, when stu- 
dents work with each other in a community project at an elderly home, they 
witness and experience something unfamiliar to them. Some may go home and 
reflect on what they witnessed, others may even discuss with each other the 
issues that impressed or confused them. In a classroom, this kind of reflection 
is rare unless by design. By the nature of experiential learning activities, it is 
common for the learners to reflect, hence better understanding the concepts, 
theories and situations to which they are exposed. Such reflection will guide 
their learning. 


Meaningful Feedback — in most experiential learning theories or frameworks, 
feedback is often dismissed. In the classroom, teachers can provide, or assign 
a person to provide feedback, resulting in a more defined process. The person 
providing feedback may be the teacher, the tutor, a peer or even the student 
themselves. In experiential learning, this is less defined. Coming back from an 
elderly home community project, discussions among students provide them with 
valuable peer feedback; students who go home and talk to their parents may also 
receive feedback from them. The feedback received is often not generated inten- 
tionally in a formulaic manner, but rather organically through the day-to-day 
experiential learning experience that students have lived. To ensure high-quality 
feedback, teachers can design experiential learning activities that include innova- 
tive feedback mechanisms such as peer mentoring. 


Mistakes are Valuable — In life, trial and error cannot be avoided. Students 
should be given opportunities to make mistakes, to learn not to fear them, but 
to find value in them instead. The process of learning is more important than the 
results. Experiential learning allows that. 


Actual Experience — Practice makes perfect — that is a fact. Each student has 
unique prior experiences and personalities, and how they interpret and inter- 
act with their experience varies. In experiential learning, teachers and parents 
have to learn how to let go and allow our students to learn and develop by 
experiencing. That is the actual value within experiential learning. 


1.8 Challenges of Experiential Learning to Different 
Stakeholders 


Following the list of benefits of experiential learning, a list of challenges is pro- 
vided below for grant reviewers and different stakeholders to make aware of 
the challenges that they may face if experiential learning activities are to be 
implemented. 
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Challenges for Universities (Fenton and Gallant, 2016; Pan, Seow, 
and Koh, 2018) 


Experiential learning is often resource- and labour-intensive; universities 
need to invest in hiring relevant professionals, develop partnerships and pro- 
vide support for professional development; 

Significant structural changes in the curriculum, which leads to a large 
amount of administrative work; 

Safety measures and insurance need to be put in place; 

Tensions between the call for the development of experiential education and 
the emphasis on research and publications. 


Challenges for Teachers (Austin & Rust, 2015; Fenton & Gallant, 2016; 
Pan et al., Koh, 2018) 


Accommodate changes in long-standing teaching practices; 

Workload often concentrates on the organisation of the activity; 

Constant and unexpected changes may arise in the course of the activity; 
More guidance for students is needed; 

More time commitment and workload required, especially when the curric- 
ulum is first implemented; 

Difficulties in ensuring the quality of experiential learning programmes and 
the delivery of learning outcomes; 

Tensions between research and heavier workload for experiential 
education; 

Difficulties in developing mechanisms for assessment and evaluation, such 
as setting assessment rubrics and maintaining consistency; 

The teacher's role is different in experiential learning activities and they have 
less control. 


Challenges for Students (Fenton & Gallant, 2016; Pan et al. 2018) 


More time and effort required than the conventional mode of learning; 
Frustration with new teaching and learning styles, different learning envi- 
ronments and unfamiliar assessment standards; 

More responsibility for their own learning; 

Some students may be unable to understand the benefits of experiential 
learning or why they are participating in some activities; 

Students (both high or low performance) often dislike group work and col- 
laboration (Chang & Brickman, 2018; Matthews, 1992). 


Benefits seem to outweigh challenges for different stakeholders, particularly for 
students, to enhance learning. The weight of each challenge may differ depend- 
ing on the individual university, teacher and student. Only the individual stake- 
holder can judge this weight and consider ifthe cost of challenges outweighs that 
of benefits for them. 
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Conclusions 


In its simplest form, 
“Experiential learning is learning by doing.” 


With the varied definitions and experiences of experiential learning interpreted 
by the different stakeholders, the author hereby concludes with some common 
features and effective practices of experiential learning to help readers develop 
their own understanding of experiential learning. 


The Must-Have Features of Experiential Learning 


Doing must be part of the learning process; 

Experiencing must be part of the learning process; 

Observing must be part of the learning process; 

Reflecting must be part of the learning process; 

Relevancy to the real world must be part of the learning process; 


ON Ui 4 Q A — 


Previous experiences and knowledge must be considered and be part of the 
learning process; 

7  Allowing for trial and error must be part of the learning process; 

8 Experiential learning is a continuous process. 


Recommended Practices in the Implementation of Experiential Learning 


1 The learning outcomes are not set in stone. Students should be allowed to 
identify new knowledge and skills, reflect on them as they learn, rethink, 
relearn, reapply and re-experience; 

2  Experiential learning should allow students to have ownership and respon- 
sibility for their self-directed learning; 

3 Holistic competencies are often best developed within experiential learning; 

thus, teachers should embrace them in the design of the activities; 

Effective experiential learning often happens outside the classroom; 

Learning may occur without the need for textbooks or lectures; 

Experiential learning changes the role of teachers into facilitators; 


` ° % + 


Opportunities to practice should be integrated into the learning and not as 

a one-off experience; 

8 The rationale for participation in the activity is important, so make it per- 
sonally relevant; 

9 Teachers should make the learning process meaningful; 

10 Iflearning is to be assessed, meaningful outcomes and clear criteria need to 
be in place; 

11 Without feedback integrated into the learning process, the learning will be de- 

layed and not constructive to student learning and competency development. 
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According to Chapman et al. (1995, p. 243): 


“Simple participation in a prescribed set of learning experiences does 
not make something experiential. The experiential methodology is not 
linear, cyclical, or even patterned. It is a series of working principles, all 
of which are equally important or must be present to varying degrees at 
some time during experiential learning. These principles are required 
no matter what activity the student is engaged in or where the learning 
takes place”. 


Questions to Ponder 


e Why would you use experiential learning to enhance your student's 
learning? 

e What are the main purposes of your experiential learning activity? 

e What challenges might you and your students encounter? 

° Does your experiential learning programme have must-have features? 


Personal Reflection 


In this chapter, I provided considerable information from the perspectives 
of different stakeholders to demonstrate what experiential learning is or is 
not. Personally, I don’t think the definition or the terminology used for 
experiential learning from one renowned scholar or another should set the 
boundaries of what learning should be or determine how teachers design 
their experiential learning course or project. Like experiential learning, 
education needs to constantly reflect, relearn, react, reinvent, reform 
and reapply to prepare future teachers, students and citizens. Future uni- 
versities cannot work in isolation on research or teaching; these university 
missions require students, industries and the communities to work closely 
to create the future society. 


Note 


1 Holistic competency is defined by the authors (Chan & Yeung, 2020) as an umbrella 
term for generic skills (e.g. teamwork, self-management and creativity), attitudes (e.g. 
resilience and hardworking) and virtues (e.g. respect, honesty and dignity). 
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2 Experiential Learning 
Theories and Frameworks 


With academic knowledge, there is deep and surface approach to learning, with 
holistic competency, there is engagement and avoidance approach to development. 
— Chan, CKY 


Introduction 


Understanding the history of experiential learning and how it came about in 
education may sometimes provide us with thought-provoking perspectives. The 
interrelationship and connectedness between frameworks, built upon a combi- 
nation of theories, can help teachers and scholars find their own paths in design- 
ing their curricula or research studies. In fact, the general concept of learning 
through experience has been around for a long time — philosophers and scholars 
from East to West have long identified that “to learn is to do.” The renowned 
Chinese philosopher Confucius (520 BC) presented us with an important mes- 
sage about learning: 


“T hear and I forget. I see and I remember. I do and I understand.” 


Relating this simple quote to some of our current teaching and learning ap- 
proaches tells us that if students passively listen to a lecture, most knowledge 
(in a colloquial sense) will go in one ear and out the other. Watching a video or 
a PowerPoint presentation may make things a bit more interesting and memo- 
rable, but for learning to truly take place, experience and practice are needed. 
Aristotle also wrote in the Nichomachean Ethics over 2,300 years ago (i.e. 
about 350 BC): 


“for the things we have to learn before we can do them, we learn by doing 
them", 


which echoed Confucius's concept. Another Chinese philosopher, Zhu Xi 
(1130-1200), crystallises the essence of experiential learning: 
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“When you know something, but you don’t act on it, your knowledge of 
it is still superficial. After you’ve personally experienced it, your knowledge 
of it will be much clearer and its significance will be different from what it 
used to be.” 

Zhu Xi (Gallant Ho Experiential Learning Centre, 2021) 


Zhu Xi’s interpretation resembles what experiential learning portrays in educa- 
tion today. 

“Learning by doing” is the central concept in the education theory of the influ- 
ential American educationist, John Dewey. Dewey is known as the father of the 
progressive education movement. Part of the progressive educational movement 
is constructivism. Constructivism as a paradigm posits that learning is an active, 
deep and constructive process in contrast to the rote and didactic approach. Dewey 
hypothesised that learning is the construction of experiences and constructivism 
is the theory of learning in which the user/learner/participant creates personal 
learning outcomes through their experience and reflection on previous experi- 
ences. He believes that experience, inquiry and reflection are the key elements of 
experiential learning. Dewey also pushed for education reforms to be grounded in 
real authentic experience, as he wrote in the following (Dewey, 1938), 


“If you have doubts about how learning happens, engage in sustained in- 
quiry: study, ponder, consider alternative possibilities and arrive at your be- 
lief grounded in evidence.” 


Other educational psychologists such as Kurt Lewin, Jean Piaget, Lev Vygotsky, 
Carl Jung, Paulo Friere and William James also place experience as the cen- 
tre of their learning theories. Piaget (1999) emphasises learning as a lifelong 
process of discovering knowledge, and the assimilation and accommodation of 
learning from experience and knowledge. Lewin (1946) coined the term “action 
research” and described it as a “circle of planning, action and fact-finding about 
the result of the action.” 


2.1 Dale’s Cone of Experience 


Edgar Dale theorised a model known as “Cone of Experience” (Dale, 1946). 
His model resonates with Confucius’ philosophy discussed above such that 
learners retain information best when they “do” things during concrete, direct 
and purposeful experiences (experience closest to real life). 

Learning experiences according to Dale’s Cone of Experience can be cate- 
gorised into three modes in a progression (i.e. enactive, iconic and symbolic; 
Bruner, 1966). The Cone visualises how different learning experiences can be 
categorised and ranked in terms of their concreteness (at the base) and abstract- 
ness (at the peak). “Concreteness” and “abstractness” here refer to how directly 
learners experience their learning. When learners experience their learning more 
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Figure 2.1 Dale's cone of experience. 


directly (e.g., doing something and sensing something using five senses), the 
learning is more “concrete.” In contrast, when learners learn more distantly 
(e.g., relying only on reading reference texts), the learning is more “abstract.” 
Dale's Cone of Experience is shown in Figure 2.1. 


Learning experiences can then be broadly categorised into three groups, from 


the most concrete to the most abstract: 


i 


Learning by doing means that learners learn through hands-on tasks. 
Learning of this kind is closely related to real experiences in which learn- 
ers can actively engage. For example, if an instructor wants the learners to 
understand more about low-cost water filtering systems, the instructor can 
provide water filtering equipment and ask them to conduct an experiment 
to distinguish between a high- and low-cost filtering system. The instructor 
can guide this experiment by inviting students to solve a real-life problem 
such as: “Poor water quality in developing countries is threatening their 
people’s health. What can we do?” 

Learning through observation means that learners learn through observing 
how people do something. Learning of this kind is more distant from real 
experiences but still preserves some elements of authenticity to which the 
learners can easily relate. For example, the same instructor can show the 
learners documentaries on how water filtering systems are built in develop- 
ing countries and ask the learners to improve such systems based on what 
they have observed. 
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iii Learning through abstraction means that learners learn only through read- 
ing visual symbols (e.g., written texts) and listening to verbal symbols (e.g., 
spoken language). Learning of this kind is most distant from real experi- 
ences, and it is, therefore, more difficult for learners to actively engage in 
their learning experiences. For example, the instructor can ask the learners 
to read a textbook chapter about low-cost water filtering systems and ask 
questions to check their understanding. 


2.1.1 Criticisms of Dale’s Cone of Experience (CoE) 


The Cone of Experience (CoE) has never been properly proven and is often chal- 
lenged by scholars who tried to debunk “the corrupted CoE,” insisting that the 
pyramid conveys no rank order (Lee & Reeves, 2007; Molenda, 2003). How- 
ever, I personally find this “cone” very intuitive as the visual model is simple and 
clear for teachers and learners to understand and can be applied broadly. 


2.2 Kolb’s Experiential Learning Theory 


Building on the theories of Dewey, Lewin and Piaget, Kolb explored the pro- 
cesses associated with the perceptions of concrete experiences and the different 
types of learning styles associated with each process. He developed a holistic 
model of the experiential learning process that is known as the Kolb’s Experien- 
tial Learning Theory. 

Kolb’s theory has widely informed programme and curriculum development 
in various institutions across the world. It has provided a general framework for 
designing fieldwork in many disciplines (e.g., Abdulwahed & Nagy, 2009; Hea- 
ley & Jenkins, 2000; McNamara, 2015; Valentine & Speece, 2002). These pro- 
grammes have incorporated elements using a variety of approaches, such as group 
work, simulation, application of theories and reflection. These approaches take 
students through the experiential learning cycle, which, in turn, improve their 
learning outcomes related to both academic knowledge and holistic competency. 

Kolb’s theories consist of three main parts, all of which are based on con- 
structivist ideology. First are the principles that guide the theory. There are six 
principle assumptions of learning that Kolb built upon a series of work from 
educational scholars that form the basis of his experiential learning theory. Sec- 
ond is the experiential learning theory itself, which involves four stages through 
which learners progress by assimilating knowledge and competencies during 
their learning experiences. The last part is about four basic learning styles. 


2.2.1 The Six Principles of Experiential Learning 
The six principles of experiential learning include: 
e Learning as a process 


e Learning is relearning 
e Emotional reflection 
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e Holistic learning 
e Environment learning 
e Constructivism 


Learning as a process. Learning is a continuous process grounded in experience. 
Knowledge is continuously derived and tested out in learners’ experiences. It is 
best to assume that learning is without an end. Kolb does not believe in look- 
ing at learning in terms of outcomes. Instead, he believes that educators should 
focus on improving the students’ learning experience by scaffolding students 
to empower them towards obtaining information and developing a good grasp 
of the material. In sum, learning is best conceived as a process, not in terms of 
outcomes. 


Learning is relearning. Students must learn, refine, and relearn, as learning is a 
continuous process with no starting point and no end point. 


Emotional reflection. Learning often requires resolution of conflicts between 
opposing models of adaptation to the world. That is, learners must move be- 
tween opposing modes of reflection, action, feeling and thinking. Thus, to 
truly learn, internal and external agreements, conflicts and disagreements may 
exist and drive the learning process. Learners have to consider multiple per- 
spectives, critique them and finally, make their own judgement based on their 
previous experiences or preferences. While this may often result in an internal 
struggle in the learners’ own beliefs, it is learning in progress. 


Holistic learning. Learning is a holistic process. It is not simply a cognitive 
process in which students gain knowledge but also encompasses how the learn- 
ers behave, think, feel and perceive their feelings, thoughts and reflection. For 
example, from the emotional reflection principle, the process of internal and 
external struggles and experiences can often uncover the holistic person from 
within the learner. 


Environment learning. Learning results from synergetic transactions be- 
tween the person and the environment. The learner does this by merging new 
experiences into established concepts, which allows the learner to grow and 
adapt to new experiences in novel environments. Learners may perceive the 
same event differently based on their previous experience. Or alternatively, the 
same student may experience the same situation differently due to different 
environments and contexts (e.g., different people, different places and different 
interactions). Environment changes the perceived learning. 


Constructivism. Learning is the process of creating knowledge, which is un- 
derstood as the result of the transaction between social knowledge and personal 
knowledge. As every learner has different prior experience and their experiences 
are constantly transforming depending on their emotions, attitudes or environ- 
ments, constructive learning inspires unique viewpoints, processes and learning 
for each learner. 
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2.2.2 Experiential Learning Theory (ELT) 


Kolb (1984) postulates a four-stage recursive experiential learning cycle that 
consists of concrete experience (CE), reflective observation (RO), abstract con- 
ceptualisation (AC) and active experimentation (AE), i.e. experiencing, reflect- 
ing, thinking and acting (Kolb & Kolb, 2009, p. 297) (see Figure 2.2). Together 
with the six principles, ELT emphasises the importance of prior knowledge, en- 
gagement with concrete experience where prior knowledge can be put into prac- 
tice, reflections on the experience, forming theories about the experience, and 
engagement with new experience where the theories can be tested (Kolb, 1984). 
In layman’s terms, the cycle has four steps — you do something, you reflect upon 
that action, and based on that reflection, you conclude and modify your under- 
standing before doing that activity or related activities again to see if you have 
improved, and followed by a further reflection. Kolb has generalised the terms 
so that students can experience, reflect, think and adapt as they experience their 
learning and at the same time, transfer Kolb's theory into different disciplines 
and environments. 

According to Kolb and Kolb (2013), learning occurs in the *resolution of cre- 
ative tension among these four learning modes” (p. 8). It is an “idealized learn- 
ing cycle or spiral where the learner ‘touches all the bases'—experiencing (CE), 
reflecting (RO), thinking (AC), and acting (AE)—in a recursive process that is 
sensitive to the learning situation and what is being learned" (p. 8). In short, in 
an ideal situation, the learner should be given opportunities and an appropriate 
environment to experience, reflect, think and act; it may not necessarily begin 
from CE, but learning will be at its best if all the four elements in the cycle are 
executed. 


Concrete Experience 
Feeling 


Perception 
Continuum 


Active Processing Reflective 
Experimentation Observation 
Doing Watching 


Abstract 
Conceptualisation 
Thinking 


Figure 2.2 Kolb's experiential learning theory and learning styles. 
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2.2.3 Kolb’s Learning Styles 


The learning cycle proposed by Kolb is based on two continuums forming four 
quadrants. The Processing Continuum, portraying the duality of active experi- 
mentation and reflective observation, refers to the learner’s personal approaches 
to a task. The Perception Continuum, which captures the duality of concrete 
experience and abstract conceptualisation, refers to the learner’s emotional re- 
sponse. The four distinct stages along the continuums, as mentioned, are active 
experimentation, reflective observation, concrete experience and abstract con- 
ceptualisation. Kolb views the learning process as people moving along the two 
continuums, and the effectiveness of learning depends on the balance between 
these stages. In ELT, Kolb believes that a learner cannot perform on both ends 
of a continuum, and the learning styles of learners would fall into either one 
of the quadrants. Referring to the four stages identified, Kolb proposed four 
learning styles: 


e Diverging (concrete, reflective) — This learning style emphasises an inno- 
vative and imaginative approach in learning and favours observation rather 
than action. Learners of this style look at things from different perspectives, 
gather information and solve problems through imagination. 

e  Assimilating (abstract, reflective) — This learning style compiles different ob- 
servations and thoughts to form an integrated view of an event. Learners of 
this style adopt a concise and logical approach and prefer clear explanations 
in learning over practical opportunities. 

° Converging (abstract, active) — This learning style emphasises the practical 
application of ideas and problem-solving. Learners of this style prefer tech- 
nical tasks and look for practical applications of ideas. 

° Accommodating (concrete, active) — This learning style is defined by solving 
problems through trial and error. Learners of this style rely on intuition and 
take practical and experiential approaches to solve problems. 


In designing a cycle of experiential learning, Kolb's learning theory sets out four 
learning styles based on the four-stage cycle. The learning styles are not fixed 
personality traits, but rather behavioural patterns in learning, which could be 
seen as learning preferences. The learning styles theorise how people prefer to 
learn and help learners identify themselves within the learning cycle. 


2.2.4 Criticisms of Kolb’s ELT 


Kolb's ELT is one of the most renowned experiential learning models used in 
education today, and there are many adaptations and applications of the model. 
The model is widely adapted partly due to the absence of other simple and clear 
experiential learning models. In fact, many scholars have criticised ELT for being 
too narrow, underdeveloped, and loosely explained to be employed effectively 
in curriculum design (Heron, 1992); this may explain why so many different 
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definitions and types of experiential learning existed today. Rogers (1996, 
p. 108) pointed out that “learning includes goals, purposes, intentions, choice 
and decision-making, and it is not at all clear where these elements fit into the 
learning cycle.” 

The circular model often gives the impression that the stages are equal in 
time and weight. Bergsteiner et al. (2010) argued that Kolb’s model is fraught 
with modelling issues such as graphic syntax errors, categorisation problems, and 
misconstrued bi-polarities; while Boud et al. (1985) remarked that the key stage 
of observation and reflection is not sufficiently explained to illustrate what reflec- 
tion entails. Further, the stepwise cycle is believed to be too simplistic to capture 
the reality of complex experiences (Ord & Leather, 2011). As Hovelynck (2001) 
demonstrated in a study of outdoor education, the practice theories educators 
develop can more accurately reflect their work; whereas a stepwise, cyclical pro- 
cess of experiential learning “might actually inform practice only superficially” 
(Seaman, 2008, p. 9). 

In addition, Kolb’s ELT appears to be primarily centred on cognitivism, 
meaning that an individual is viewed as a separate entity from his/her environ- 
ment (Holman et al., 1997). The cognitivist perspective means that the theory 
overemphasises individual experience at the expense of social relations, and thus 
it does not account for how unequal power relationships can affect the way peo- 
ple talk about their experiences (Vince, 1998). As Fenwick (2000) pointed out, 
Kolb’s ELT lacks the ability to explain learning in a social context where indi- 
vidual knowledge often converges and co-emerges with collective knowledge. In 
experiential learning, social interactions of a learner are very important to the 
development of self, thought and learning, particularly for young adults. Cul- 
ture, prior experience, social pressure and yearnings could greatly impact how 
learning is developed and enhanced. 

With regard to the learning styles, the learning style inventory based on ELT 
has also been criticised for lacking validity and reliability (Coffield et al., 2004; 
Freedman & Stumpf, 1980). In principle, it is always difficult to generalise or 
characterise people into types, learners have “profiles (or patterns) of styles, not 
just a single style” (Sternberg, 1997, p. 83). There are many factors that may 
change the approach a learner takes; in short, human anthropology is complicated. 


2.3 Experiential Learning Models Targeting the 
Development of Holistic Competency 


Experiential learning often targets not only academic knowledge but also the 
development and enhancement of holistic competency. Holistic competency is 
difficult to develop in conventional classroom environments, but is relatively 
easier to develop in experiential learning activities where students make mean- 
ing out of their participation (Chan, 2012, 2012b). I collected data from over 
4,500 students across multiple universities in Hong Kong, and 86% of them 
agreed that holistic competency is better developed through out-of-classroom 
activities (e.g. internship, field trip, living in halls, joining societies and part-time 
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Figure 2.3 Tag cloud visualisation of the preferred methods for the development of com- 
petency from students’ perspectives. 


work) than in-class activities. Figure 2.3 shows a tag cloud visualisation present- 
ing the preferred methods of competency development from students’ perspec- 
tives. Table 2.1 shows a classification of the types of learning activities (in-class, 
out-of-class and extra-curricular) that are often used for the development of ho- 
listic competency in university. 


2.3.1 Challenges in the Development and Implementation of 
Holistic Competency 


My team and I (Chan et al., 2017) conducted a literature review on 56 jour- 
nal papers published in the last two decades and raised three main areas that 
pose the most challenges to developing and implementing holistic competency 
in higher education. Figure 2.4 shows a diagrammatic representation of the chal- 
lenges found. 

First and foremost, although many universities have incorporated competency 
development into their graduate attributes, there has been inadequate institu- 
tional and curriculum support. While some universities believe it is not their 
responsibility to offer holistic competency training (Dunne et al., 1997), others 
find it difficult to launch a curriculum that closely aligns with the rapid-chang- 
ing nature of these competencies (Jackling & De Lange, 2009; Jackson, 2016), 
especially in a context of the ever-increasing number of students and when large 
classes and summative assessment are preferred (Bunney et al., 2015; Yorke & 
Harvey, 2005). Additionally, teachers have not received sufficient training 
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Table 2.1 Classification of the types of learning activities (in-class, out-of-class and 
extra-curricular) for the development of holistic competency in university 


Classification of learning activities and opportunities for holistic competencies 
development 


Discipline- E.g. community service learning’; field 
specific Out-of trips*; internships*; work placements*; 
class student exchange*; student teaching’; 
E.g. capstone projects*; problem- 
based learning (PBL) *; presentations*; 
Academic In-class clinical and non-clinical 
related practicums*; laboratory*; research 
fellowship*; group projects*; workshops* 
Non- Extra- E.g. common core courses*; residential 
discipline EL education; camps; general education*; 
specific student societies and clubs; workshops 


and guest speakers; sports activities; 
career guidance; entrepreneurship- 
related activities/competition; externship; 
community service learning; field trips; 
internships; work placements. 


Non-academic related E.g. experiences in daily life; work 
experiences; interactions with friends 
and family; charity and non-profit 
organisation participation; media, 
internet and socialisation 


* Maybe credit bearing. 


with regard to developing students’ holistic competency (Bunney et al., 2015; 
Chan, 20122). 

There are also palpable operational challenges, mainly concerning the neb- 
ulous holistic competency conceptualisation, ineffective teaching pedagogies 
and difficulties in assessment. Due to a plethora of related items used inter- 
changeably with competency and a lack of a standardised list of what these 
competencies exactly include (Barrie, 2005, 2012), it is difficult to achieve a 
clear conceptual base for competency development. While research has sup- 
ported active learning and extracurricular events as effective ways to promote 
holistic competency development (Chan & Yeung, 2020; Kember et al., 2007), 
the actualisation of these pedagogies, however, often faces resistance from stu- 
dents and teachers (Chan, 2012b; Green et al., 2009). Apart from pedago- 
gies, the assessment of competency is also highly contestable. Some scholars 
criticised that holistic competency is rarely assessed and is hidden in the cur- 
riculum (Badcock, Pattison, & Harris, 2010), others questioned the validity, 
reliability, and feasibility of assessing competency (Hughes & Barrie, 2010). In 
some exam-oriented cultures, assessment may cause students to eventually lose 
interest in holistic competency if they are developed mainly for grading and 
evaluation purposes (Chan & Yeung, 2021). That being said, the dilemma is 
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that, without formal assessment, it is less likely that teachers and students will 
take holistic competency development seriously in an already tight curricular 
schedule (Chan & Chen, 2021; Chan & Luk, 2021b). 

Furthermore, various perceptions and scepticism of holistic competency from 
students and teachers also present challenges. In higher education where re- 
search outputs are often considered more vital when associated with recruitment 
and promotion, teachers tend to give more priority to research over teaching and 
are reluctant to adopt innovative teaching approaches in favour of competency 
development due to extra workload (Drummond et al., 1998). Some teachers 
even view the development of holistic competency as an item on a “checklist” 
imposed by the university (Krause, 2014). As for students, while they recog- 
nise the need to acquire academic knowledge, they tend to perceive competency 
development as “time-consuming” and irrelevant to their study (Dunne et al., 
1997; Hughes & Barrie, 2010). Even for those who do realise the importance of 
holistic competency, their undergraduate career has not effectively fostered their 
competency development (De La Harpe et al., 2000). 
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Figure 2.4 A review of literature on challenges in the development and implementation 
of holistic competency in higher education curriculum (Chan et al., 2017). 


Source: Reproduced with permission from authors. 
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To address these challenges, the authors argued that self-motivation in ho- 
listic competency development will be crucial for teachers and students, who 
should better recognise the importance of holistic competency. A more reliable 
and systematic competency assessment, reporting and recognition mechanism 
is also necessary to further promote learning and provide evidence (Chan & 
Chen, 2021; Chan & Luk, 2021a; Chan & Luo, 2021). Meanwhile, increased 
institutional support is required to ensure that teaching pedagogies and student 
experience are aligned with holistic competency development. 


2.4 Holistic Competency Development Framework 
(HCDEF) - Approach to Develop 


As discussed above, holistic competency is often developed through extracur- 
ricular events. These events are typically non-credit-based and do not count 
towards summative assessment. To explain students’ approach to developing 
holistic competency, a Holistic Competency Development Framework (HCDF) 
(see Figure 2.6) was deduced and validated (Chan & Yeung, 2020). The frame- 
work has five key components that are fundamental to students’ approach to 
holistic competency development: (1) learner character; (2) rationale for develop- 
ing; (3) students’ actual learning experience, and their perceptions and interpre- 
tations based on their experience; (4) students’ approaches to developing holistic 
competency; and (5) learners’ development of holistic competency as outcomes 
(Chan & Yeung, 2020). 

The HCDF is an adaption of Bigg’s 3P model (1987). It was argued that 
traditional learning processes such as the 3P model do not apply to holistic com- 
petency development, because learners who are deep learners in the academic 
context do not necessarily become deep learners in holistic competency educa- 
tion. Thus, the words “deep” and “surface” with respect to academic knowledge 
are unsuitable in the holistic competency context. Often, it is those deep learn- 
ers who are strong academically but tend to avoid the development of holistic 
competency — they may not engage in experiential learning activities as they feel 
that these engagements are obstacles to their real goals (i.e. grades) in academia. 
Consequently, a new term — “Approach to Develop” (Chan & Yeung, 2020) was 
coined — this is as opposed to “Approach to Learn,” and tailored for conceptu- 
alising and quantifying learners’ engagement in experiential learning leading to 
the development of holistic competency. The constituents “deep” and “surface” 
approaches associated with 3P from Biggs and Tang (2011), Entwistle (1988), 
Ramsden (1992) and Prosser and Trigwell (1999) in student’s “Approach to 
Learning” are replaced by “engage” and “avoid” in students’ “Approach to De- 
veloping.” A learner is considered as an “engager” if they take the opportunity 
to engage in an activity; the development of competencies is welcomed and prac- 
ticed. A learner is considered as an “avoider” if they avoid an activity and thus 
leave little room for competency development. 

When teachers or coordinators design an experiential learning course or activ- 
ity, the HCDF framework helps the course designer through a student-centred 
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Table 2.2 Learn vs develop student approaches 


Academic Knowledge vs Holistic Competency 
Learn Develop 

Approach to learn vs Approach to develop 
“Deep” and “Surface” “Engage” and “Avoid” 


Figure 2.5 Competency development weight scale with approach to develop and ap- 
proach to learn (Drawing by Aisling Colloton). 


learning design process. One particularly important element from the HCDF 
framework is students’ rationales or motives for developing holistic compe- 
tency, and five rationales have been identified, namely; meaning-driven, ca- 
reer-driven, enjoyment-driven, course-driven and family-driven. Course 
designers are encouraged to design their experiential learning courses with 
these rationales in mind, which motivates students to join and engage in 
activities. 

Unlike academic knowledge, holistic competency must be developed by ex- 
perience. For example, leadership skills cannot be learnt through attending a 
lecture or reading a book; the learner must be given opportunities to observe 
and experience what leadership is. Hence, the word “learn” can be used to de- 
scribe academic knowledge acquisition, whilst “develop” is more suitable for 
describing holistic competency education. Table 2.2 shows the constituents of 
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Presage Process Product 


Learner Character 
(e.g., personality, 
learning motives, 
prior experience) 


Actual Learning 

Experience Student’s 

(e.g. learning Perception & 
activities and Interpretation 
experience, work Based on the 

and assessment, Learning Experience 
workload, quality of (e.g., perception of 
supervision and their experience and 
interaction with reflection) 


Student’s Approaches 

to Develop - Degree 

of Engagement 

Engage and Avoid Competency 
continuum (e.g., Development 
reflection, feedback Outcomes 
seeking, meaningful 


Rationale/Motives social interactions) 


for development 

(e.g., 

- meaning-driven 

- career-driven 

- enjoyment-driven 

- course-driven A learner is considered as an "engager' if he/she takes the opportunity 

- family-driven) of engaging in an activity; the development of competencies is 
welcomed and practiced. A learner is considered as an "avoider" if 

he/she avoids an activity and thus there is little room for competencies 

development. 


Note: Assumption is that students who go through the process described 
by this framework are already willing to participate the learning activity. 


Figure 2.6 Students’ approaches to holistic competency development framework (HCDF) 
(Chan & Yeung 2020). 


Source: Reproduced with permission from authors. 


students’ approaches for academic knowledge as opposed to holistic competency. 
Figure 2.5 shows an impression by Aisling Colloton showing the Competency 
Development Weight Scale. 


2.5 “Chain of Mirrors — The Metacognitive 
Development Model” - A Practical Guide for 
Curriculum Design in Holistic Competency 


In the absence of a more concrete model to guide curriculum design for holistic 
competency development, I developed a model known as the “Chain of Mirrors — 
the Metacognitive Development Model” (Chan, 2016; Chan, 2019). It has been 
developed to delineate the development of holistic competency and provide clearer 
guidance for teachers to design effective experiential learning activities in order to 
optimise learners’ development of holistic competency (see Figure 2.7). 

The model provides important “mirrors” that aid teachers in incorporating 
experiential learning activities and opportunities into their curriculum, enabling 
effective development and enhancement of learners’ competencies. The model 
was developed from the conceptualisation of the experiential learning process, 
the metacognitive processes, self-concept theory and student-centred behaviours 
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Rational | Self- l | | Experience | (ergo | 
Motives Awareness | What have | What have | 


Why am | What do | done and 
doing this? already | [ contributed? 
know? 


learnt and 
developed? 


Appraisal | 


And Reflection 


Feedback How do l feel? 
How did I really What would I 
do from my eyes do differently 

and others? next time? 


Figure 2.7 Chain of mirrors — the metacognitive development model a practical guide for 
curriculum design for holistic competency. 


involved in learning. These student-centred behaviours include rationales, 
self-awareness and expectations to engage or avoid an activity; as well as trial and 
error in the implementation of holistic competency activities. In the previous 
section, the HCDF framework explains the importance of students’ approach to 
develop, as students engage in holistic competency development differently from 
their approach to learn academic knowledge. 

In fact, to truly enhance students’ development of holistic competency, the 
power of metacognition should not be underestimated. Metacognition is the 
process of thinking about one’s own thinking, which involves the processes of 
planning, tracking and assessing one’s own thinking and achievements. Flavell 
(1976) describes it as follows: “Metacognition refers to one’s knowledge con- 
cerning one’s own cognitive processes or anything related to them, e.g., the 
learning-relevant properties of information or data.” The Chain of Mirrors em- 
ploys the key concepts of metacognitive knowledge and regulation to monitor 
holistic competency. The term “mirror” is used to illustrate how the learner 
constantly looks into a mirror at different stages of the development process. 
There are seven mirrors in the Chain of Mirrors and each of them is explained 
below. 


2.5.1 The Rationales/Motives Mirror 


In academic learning contexts, learners typically take a course or a programme 
for career pursuits. But for holistic competency which is not closely related to 
any disciplines, students’? motives to participate in a learning activity can be 
rather diverse (Chan & Yeung, 2020). Based on a large-scale study, learners’ 
motives were driven and influenced if the activity is (1) meaningful, (2) fun, 
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(3) career-related, (4) course-related, (5) family-influenced, as explained in the 
HCDF section (Chan & Yeung, 2020). The value of participating in a learning 
activity is determined largely by learners’ motives to develop holistic compe- 
tency, which, to some extent, also hinges upon their prior experience (impacting 
preference) and self-awareness, which is another mirror to be introduced. 


2.5.2 The Self-Awareness Mirror 


Understanding oneself is an important first step in formulating learning goals, 
and is thus an important mirror of the model. For a person to engage in such 
development, they must not only have a good understanding of their own 
strengths and weaknesses, but they must also be able to recognise that “I am a 
learner? (Kolb & Kolb, 2009, p. 297). One ofthe important factor in developing 
holistic competency is that learners are often unaware of their own strengths and 
weaknesses regarding the different competencies. The Chain of Mirrors model 
highlights a need to offer opportunities to enhance self-understanding so that 
learners can look through the *mirror" and become aware of their strengths and 
weaknesses in terms of their skills, feelings and behaviours. Self-assessment sur- 
veys, learning contracts (to learn more about learning contracts, go to Chapter 4) 
and SWOT analyses are common tools to enhance self-understanding. As learn- 
ers have different prior levels of competencies, involving them in making deci- 
sions about their learning needs through self-evaluation can encourage them to 
take a more active role in directing and reflecting upon their learning, which is 
particularly conducive to holistic competency development. 


2.5.3 The Expectation Mirror 


When learners decide to take part in a learning activity, they would try to un- 
derstand what they expect to learn, set goals and decide on the approach they 
would take to achieve these goals. These are important metacognitive skills for 
learning (White & Frederiksen, 1998), relating to both procedural knowledge 
and strategy knowledge in metacognitive knowledge (Flavell, 1985). If learn- 
ers have a well-developed learning self-identity, they would make the best use 
of the opportunities offered by the learning activity. One way to help learners 
clearly identify and set expectations is through a learning contract. Based on 
the self-evaluation of their strengths and weaknesses in the previous mirror, the 
learning contract provides a set of *standards" or "criteria" learners themselves 
develop to self-challenge, assess and reflect. 


2.5.4 The Experience Mirror 


Experience is central to the learning process whereby learners with a clear learn- 
ing self-identity can explore, act and reflect while going through rounds of trial 
and error to create new skills and knowledge. It is therefore a core element of 
the Chain of Mirrors model. As mentioned previously, holistic competency is 
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best developed through experience. For university students, for example, we 
have found that they strongly perceived out-of-classroom and authentic activi- 
ties, such as internship and community services, as important means to enhance 
their holistic competency (Chan et al., 2017). Learners should therefore be of- 
fered opportunities to participate in experiential activities. These opportunities 
should also be carefully designed to align with learners’ rationales which deter- 
mine their “approach to develop” — whether to avoid or engage (Chan & Yeung, 
2020). Learners will not continue to engage in the provided opportunities if the 
experience is negative. 


2.5.5 The Learning Mirror 


This mirror assists learners in thinking about what they have learnt in the expe- 
rience and evaluating their learning progress against the expectations they set. 
What holistic competency have I developed? How did I develop them? Which 
competencies should I continue to work on? A learning self tends to focus more 
on the learning process and improvement needed for the future, instead of the 
immediate outcomes and results. For experiential learning and holistic compe- 
tency, the process is more important than the result, thus, the activities design 
should focus on how to improve learning during the activities and not after. 


2.5.6 The Reflection Mirror 


Reflection is instrumental in transforming experience and consolidating the ac- 
quisition of disciplinary knowledge and holistic competency. Learners constantly 
reflect upon their concrete experience, learn from mistakes and successes, seek 
new experience and deduce new knowledge. In experiential learning, reflection 
is a commonly used assessment method and is achieved by consciously focusing 
learners’ attention on what they have learnt, done, and observed, as well as how 
they have put prior knowledge into practice (more information on reflection can 
be found in Chapter 5). As holistic competency development is a long process, 
reflection — a widely used tool for tracking learning trajectories and digging deep 
down into learners’ feeling and experience — is a crucial mirror. It is said that 
powerful learners always reflect. Some questions that can aid them in the reflec- 
tive process include: Why did you do it that way? How did you feel after you had 
done that? Would you change anything you did? It is also sometimes effective to 
use artefacts such as videos or personal items to help learners reflect upon their 
feelings to facilitate holistic competency development. 


2.5.7 The Appraisal and Feedback Mirror 


The final piece of the “mirror” is appraisal and feedback. In addition to self- 
evaluation and self-assessment through reflection, it is equally essential for learners 
to receive sequential feedback from others, such as teachers, mentors, supervi- 
sors and peers as well as from the self. Effective feedback provides high-quality 
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information about learning that helps learners to clarify what good performance 
is in terms of goals and expectations (Nicol & Macfarlane-Dick, 2006), thus 
allowing the identification of gaps between their actual and expected learning 
outcomes. For learners who value social acceptance, they desire to know how 
others perceive them. In providing feedback for holistic competency develop- 
ment, an observer can be asked to target certain competencies and consider the 
behaviour, attitudes and skills exhibited by the learners in different contexts that 
demonstrate those competencies, and then provide feedback. Obviously, as de- 
velopment occurs during the learning process, feedforward would be more vital 
than feedback, although there are approaches that allow students to continuously 
reflect and develop even after the outcomes have been achieved. Teachers should 
utilise different stakeholders to provide appraisal with student-centre in mind. 

In the Chain of Mirrors model, different mirrors prompt teachers to present 
different opportunities to engage students. Self-understanding, reflection and 
appraisal can be used to guide the development of appropriate assessments for 
holistic competency with carefully designed curricula and activities. 


Conclusions 


In this chapter, I have described many experiential learning theories and frame- 
works including those that I developed. These theories and frameworks will help 
us frame our thought processes when we design activities and assessments. In 
the next chapter, I will dive deeper into the assessment in experiential learning 
including the assessment literacy both students and teachers should possess. 


Questions to Ponder 
The only opinion that matters in life is your own, do you agree??? 


Statements like the one above can be seen everywhere — with words like 
“the only opinion that matters is the one you have about yourself.” It is 
also something we like to say to our young students and children to make 
them feel confident about themselves. However, is that true? In reality, 
how our peers, our friends, our bosses and society see us are very impor- 
tant; that is how it works in society. Thus, the last mirror in the Chain of 
Mirrors reflects the vital process of appraisal and feedback from the people 
around us in supporting personal development. The colour of the mirror 
is displayed in various shades of grey to show that things are not always 
black or white in appraisal and feedback. For example, one peer may have 
a high opinion of you as a leader, while another may find your leadership 
approach too aggressive. To be successful in any particular competencies, 
the key is to receive positive opinions from different people and not just 
one type of people. How can we help our learners to enhance that in expe- 
riential learning? How can we help ourselves to enhance that? 
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Personal Reflection 


In this chapter, I presented different experiential learning theories and 
frameworks, including the Chain of Mirrors and how holistic competency 
should be developed and not learnt. While I was researching on the differ- 
ent theorists and frameworks, I reflected and pondered on a rather egoistic 
perspective. And thus, this reflection is an odd one: I wonder if some fu- 
ture scholars will write about my ideas of experiential learning and holistic 
competency “maybe” based on the Chain of Mirrors? 


Note 


1 Holistic competency is defined by the author as an umbrella term for generic skills 
(e.g. teamwork, self-management and creativity), attitudes (e.g. resilience and hard- 
working) and virtues (e.g. respect, honesty and dignity). 
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3 Assessing Academic 
Knowledge and 
Experiential Learning 


Learning outcomes of experiential learning are not rigid, they are dynamic. Stu- 
dents may not know the exact learning outcomes until the end of their learning 
process and sometimes even beyond that. 

— Chan, CKY 


Introduction: Experiential Learning and Its Learning 
Outcomes 


Knowledge economy highlights the need to incorporate both holistic competen- 
cies! and academic knowledge in education and society. With academic knowl- 
edge, intended learning outcomes are written out at large and easily recognisable, 
but with experiential learning, the problem of “which outcomes to measure and 
how to measure them? (Male & Chapman, 2005) arises. Most countries have 
adopted an outcomes-based system across all levels of their education system, but 
in a system where only one set of outcomes (the academic knowledge outcomes) 
is known, and the outcomes from the competencies set are not clear, this makes 
competencies very difficult to assess, especially when educators are expected to 
provide constructive alignment and clear criteria and grade descriptors. This chal- 
lenge in assessment can be a major barrier in experiential learning when one of the 
major benefits of experiential learning is the development of holistic competencies. 

Unlike traditional approaches to learning, the learning attainments from expe- 
riential learning can vary from one student to another student. There is a range of 
factors that influence learning attainment as shown in the Holistic Competency De- 
velopment Framework (HCDF) (Chan & Yeung, 2020) in Chapter 2, including ra- 
tionales, prior experience, expectations, actual experience and learners’ approach to 
development as well as the learning activities’ intended outcomes. These factors are 
known as *confounding variables? (Vogt, 1993). In Ewert and Sibthorp's (2009) ar- 
ticle on experiential education, they have broken down these confounding variables 
based on Vogt's definition (1993) into three main areas: precursor, concomitant, and 
post-experience. These variables include students’ prior knowledge and experience, 
demography, pre-experience motivations and expectations, group characteristics, 
situational impacts, social desirability, post-experience euphoria, etc. The influence 
of confounding variables may affect the learning gains of experiential education and 
are varied and unpredictable, hence difficult to establish and measure (Qualters, 
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2010). Thus, assessment designs should account for these variables to ensure validity, 
reliability and impact (Van Der Vleuten & Schuwirth, 2005). Table 3.1 shows Ewert 
and Sibthorp’s ideas of confounding variables for experiential education against the 
varying factors from the HCDF in the three stages for holistic competencies, note 


the similarities. 


Table 3.1 Ewert and Sibthorp’s and Chan and Yeung's HCDF - the confounding 
variables and the varying factors 


Experiential Education: Ewert and Sibthorp’s Confounding Variables (2009) 


Precursor (Before the 
Experience) 


Concomitant (During the 


Experience) 


Postexperience (After the 
Experience) 


Prior knowledge and 
experience 


Demographics 


Pre-experience anxiety 


Pre-experience 
motivations and 
expectations 


Self-selection into a 
specific program/ 
course or experience 


Course specifics 


Group characteristics (and 


interactions) 


Situational impacts 
(unanticipated events 
that occur during the 
experiential learning 
experience) 

Frontloading for 
evaluation (refers to 
instructors or other 
participants that 
may influence the 
experience) 


Social desirability (the tendency 
of the learner to respond to the 
learning that is deemed more 
acceptable or desirable despite 
their true feelings) 

Post experience euphoria (a 
feeling of excitement, positive 
affect, sense of accomplishment 
after the experiential learning) 

Post experience adjustment 
(the time for the learner to 
adjust back to normal life 
after the experiential learning 
experience) 

Response shift bias (evaluation 
of the learner’s perception 
changes over time) 


Holistic Competencies: Chan and Yeung’s HCDF (2020) 


Presage 


Process 


Product 


Learner Character (e.g., 


personality, prior 
experience) 


Rationales/ Motives for 


Learning 


Student’s Perception & 
Interpretation Based 
on the Learning 
Experience 


Student’s Approaches to 


Develop - Degree of 
Engagement 


Actual learning experience 
(e.g. learning activities and experience, 
environments, work and assessment, workload, 


quality of supervision and interaction with peers) 


Competency Development 
Outcomes 
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Depending on the factors named above, the type of experiential learning that 
a student chooses to engage in also frames their experience differently. Apart 
from confounding variables, disciplinary differences and the types of experi- 
ential learning activity a student partakes in, the assessment organised by the 
coordinator or teacher may differ, too. A capstone project may combine a pres- 
entation, thesis report, the actual product and monthly supervisor commentary 
as assessment; a common internship may use a weekly log, a reflective report 
and an employer performance appraisal form; while a social entrepreneur project 
may require students to compete for funding with a business plan and client’s 
performance report. All that would also depend on the intended learning out- 
comes and the purposes of the assessment for the particular activity; there is not 
a one-size-fits-all definition. 

As this book focuses on assessment, we will start by providing the basis of 
assessment for traditional learning, and progress to discuss the assessment of 
experiential learning. 


3.1 Assessment in Academic Knowledge 


Assessment is an on-going evaluation process aimed at understanding and im- 
proving student learning by measuring learning outcomes in knowledge, skills, 
attitudes and beliefs. It has four main purposes, namely (a) judging achieve- 
ment, (b) protecting and maintaining academic standards and quality assurance, 
(c) managing accountability for the public and for funding and (d) promoting 
student learning. 

In a student-centred outcome-based approach to learning academic knowl- 
edge, there are three components that make up the curriculum design — the 
learning outcomes, the pedagogy and the assessment (see Figure 3.1). The learn- 
ing outcomes are associated with the students’ objectives at heart — what do we 
want our students to learn? The pedagogies are associated with the learning 
activities — what activities help students achieve those learning outcomes? And 
the assessment — how do we know that the students have learnt? From students’ 
perspective, assessment defines the curriculum and influences students! approach 
to learning (Ramsden, 2003). Their perspectives towards assessment can deter- 
mine how they wish to learn, whether it is deep or surface (Bloxham & Boyd, 
2007; Elton & Laurillard, 1979; Medland, 2016; Prosser & Trigwell, 1999), 
making assessment design vital. 


3.1.1 Diagnostic, Formative, Summative and Learning-Oriented 
Assessment 


The purposes of assessment guide the decision on types of assessment. There 
are four main types of assessment, namely diagnostic, formative, summative 
and learning-oriented assessment. Conventionally, the first three types can be 
considered as an assessment before, during and after learning, while learning- 
oriented assessment may occur at any time. 


42 Assessing Academic and Experiential Learning 


Learning Outcomes 


What do you want your 
students to learn? 


Teaching and Learning Assessment 
Activities Methods 


What types of activities will 
help your students to learn? 


How will you know your 
students have learned? 


Curriculum 
List of topics to be covered 


Figure 3.1 Outcomes-based approach to student learning (academic knowledge). 


Diagnostic assessment assesses students’ prior knowledge, skills, strengths, and 
weaknesses prior to their learning. Norm-referencing is often associated with this 
type of assessment as it compares the student’s prior knowledge against the norm 
student group. It is also often employed for the allocation of students into elite 
class or groups. Diagnostic assessment is also used to identify difficulties that 
learners might have, in order to introduce appropriate learning interventions. 


Students’ Questions Associated with Diagnostic Assessment 


e What level am I at, compared to my class? 
e Will I be good enough to be in group A? 
e What are my current strengths and weaknesses? 


Teachers’ Questions Associated with Diagnostic Assessment 


e What knowledge do my students already have? 

e Should I separate the best students and put them together into the same 
group or class? 

e Should I introduce an introductory/advanced class for the weaker/stronger 
students? 


Formative assessment, also known as Assessment for Learning (AfL) assesses 
students! performance during learning and often occurs regularly during the 
learning process in order to provide feedback for improvement. Criterion-reference 
is often associated with this type of assessment to measure students’ performance 
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against goals, or specific standards. The term formative assessment was introduced 
by Michael Scriven in the 1960s’ to make a distinction from summative assess- 
ment. Black and Wiliam (1998) explained the term formative assessment in this 
light, “[an] assessment becomes formative assessment when the evidence is actually 
used to adapt teaching to meet student needs” (p. 140). Wiggins (1998) rightly as- 
serted that “the aim of [formative] assessment is primarily to educate and improve 
student performance, not merely to audit it” (p. 7). 


Students’ Questions Associated with Formative Assessment 


e Tam lost, what do I need to do to improve? 
e How am I doing? 
° [can do this, but how can I do better? 


Teachers’ Questions Associated with Formative Assessment 


° [noticed half of my class is confused, I must not have explained the topics 
well enough, should I amend my teaching approach? 
e What will I teach them next? How will I teach them next? 


Summative assessment assesses students! achievement at the end of the learn- 
ing process and is also known as Assessment of Learning. This type of assessment 
is often associated with grades. More recently, teachers are using summative 
assessment to provide interim grading and feedback. Teachers may opt to have 
summative assessments such as a term paper or a chapter test at half-term or after 
a unit of study is completed, rather than when the entire course is completed. 
In other words, summative assessment has been implemented for formative pur- 
poses. Under an outcomes-based approach, criterion-referencing is employed. 
One of the common features of summative assessment is that it is almost always 
associated with a grade to identify if the student has performed satisfactorily. 


Students" Questions Associated with Summative Assessment 


e Have I achieved the expected level? 
e Would I get an ‘A’? Would I pass? 


Teachers’ Questions Associated with Summative Assessment 


e What levels of learning have the class achieved? 
e Most of the class has achieved the standards, will this look good enough for 
the examination board and pass the quality assurance council? 


Assessment as learning (Earl, 2003) is an assessment process in which stu- 
dents become more aware of how they learn and more responsible for their own 
learning. They also learn about their own learning and thinking processes and 
become metacognitive. Students monitor their own learning, self-question and 
reflect. Assessment as learning is a process that involves self and peer assessment, 
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along with formal and informal feedback in order for students to come up with 
solutions to problems and gain insights into their future career direction. 


Learning-Oriented Assessment (Carless et al., 2006b) is when the assessment 
becomes part of the learning. Carless (2007) coined the term “learning-oriented 
assessment,” and this term does not differentiate between summative or formative. 
It aims primarily at the purpose of assessment for promoting student learning. Car- 
less asserts that learning-oriented assessment task should contain three elements; 
(a) the task should embrace learning, (b) it should involve students as assessors for 
both themselves and peers, (c) tasks should have feedback processes built in (pref- 
erably feedforward to allow the opportunity for re-action). Students monitor their 
own learning, ask questions and use a range of strategies to decide what they know, 
can do and progress, and how to use assessment information for new learning. 

For teachers and practitioners, the two terms, assessment as learning and 
learning-oriented assessment, are moderately similar, as they primarily focus on 
student learning. For researchers, there may be a subtle distinction. In Torrance’s 
(2007) article, he critiqued that “the practice of assessment has moved from as- 
sessment of learning, through assessment for learning, to assessment as learning, 
with assessment procedures and practices coming completely to dominate the 
learning experience and criteria compliance replacing learning” (p. 281). 

To implement these processes of assessment successfully and effectively, a 
large part depends on the students, as students need to take responsibility for 
their own learning. Unlike diagnostic, formative and summative assessment, 
assessment as learning and learning-oriented assessment bring out the intrinsic 
motivation in students and is also process-driven where teacher takes up a guid- 
ance role. One example of such assessment is the thesis writing by PhD and 
Master's students, who self-lead, self-learn, self-reflect and often self-assess to 
become independent learners in order to create a path for their future learning. 


Students? Questions Associated with Assessment as Learning and Learning- 
Oriented Assessment 


e What do I know about this? 

e What is the purpose of learning this? 

e What learning approaches should I adopt to help me accomplish my learning 
goals? 

° Dolunderstand it, and have I accomplished the goals I set? 

° Do the assessment and feedback make sense, and how do I get better? 


Teachers? Questions Associated with Assessment as Learning and Learning- 
Oriented Assessment 


e What are my students’ goals? What are they aiming to achieve? 

e Why did they do it that way? Are there other approaches? 

e The student's current issues are similar to the ones identified in the labora- 
tory, can he/she adapt or modify them? 
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e Why don’t they try that? Is that what they want? 
° Tam unfamiliar with the student's findings and challenges in their studies, 
how can I help them? 


Table 3.2 gives a summary of the types of assessment. The information is col- 
lated from numerous sources (Dixson & Worrell, 2016; Hanna & Dettmer, 
2004; Trumbull & Lash, 2013). 


3.1.2 Balancing Formative and Summative Assessment 


Most governments and organisations appreciate summative assessment over 
formative assessment as they provide easy and logical quantitative data such as 
that generated by standardised tests like the SATs, the Hong Kong Diploma 
of Secondary Education (HKDSE), the Chinese National College Entrance 
Examination (NCEE), is known colloquially as “Gaokao,” for accountability, 
judgement and quality assurance. It is seen as objective, measurable and easy to 
benchmark. Educational researchers (Villarroel et al., 2018) are naturally critical 
of summative assessment, as traditional summative assessment approaches en- 
courage rote learning and act as a filtering system for weaker students, bringing 
little to enhance student learning. 

Educators would naturally believe students favour formative assessment be- 
cause of its inherent feedback function, but it is often the prospect of grades 
that motivates the students due to the high-stake tradition of summative exam- 
inations (Kohn, 2011). Thus, it is essential for teachers to find the appropriate 
balance between formative and summative assessment. 

There are no rules saying that a summative assessment approach cannot have 
an add-on feedback element or that a formative assessment cannot be associated 
with a grade or credit. At the University of British Columbia, Brett Gilley from 
the Department of Earth, Ocean and Atmospheric Sciences and his colleagues 
advocated for a two-stage exam (Gilley & Clarkston, 2014). In the first stage, 
students hand in an individual copy of the test, and then during the second 
stage, they confer with a small group of four to five students, and re-do the same 
exam as a team and discuss their answers, this allows them to generate their own 
feedback and involve in active learning. The result shows better subject retention 
and observed less exam stress on their students. The two-stage exam is an exam- 
ple of a summative exam with a formative focus. 

There have been ongoing contestations on summative and formative as- 
sessment, but as teachers and researchers advance in educational research and 
practices, the assessment approaches have evolved to bridge the gap, as this can 
be seen from the list of assessment approaches in Table 3.2. The distinction 
between summative and formative assessment is superficial as both types of 
assessment can provide feedback opportunities. In fact, the two types “com- 
plement each other? and *on occasion, it may make sense to use a formative 
assessment summatively or a summative assessment formatively,” depending 
on the outcomes and purposes of the assessment (Dixson & Worrell, 2016). 
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3.1.3 The Challenges in Academic Knowledge Assessment 


There are some apparent challenges when assessing academic knowledge, and 
most of these challenges have been known for decades. In the section below, we 
expanded the seven main challenges in assessment identified in Carless, Joughin, 
and Liu (2006a)’s work. Based on these challenges, we have also added in the 
practical “irritations” experienced by teachers, administrators and students in 
recent years to paint a big picture highlighting current challenges in academic 
knowledge assessment. Many of these challenges continue to emerge because old 
habits die hard. Readers should note that most of these challenges also occur in 
experiential learning assessments. 


i 


ii 


iii 


The multiple purposes of assessment 

There are many underlying purposes of assessment. Fendrich (2007) wrote 
in the Chronicle of Higher Education in a piece titled, A Pedagogical Strait- 
jacket that 


outcomes-assessment practices in higher education are grotesque, unin- 
tentional parodies of both social science and ‘accountability. No matter 
how much they purport to be about “standards” or student “needs”, 
they are in fact scams run by bloodless bureaucrats who, do not under- 
stand the holistic nature of a good college education. 


Carless et al. (2007) brought in the term “double duty” coined by Boud 
(2000) to explain this challenge - the focus on multiple purposes of assess- 
ment. The best assessment design clarifies purposes, although some assess- 
ment methods may serve several purposes. 

The mixed-perception of assessment 

Both teachers and students have their own perceptions of the purposes of 
assessment. According to Brennan (2015), *assessment is an emotionally- 
laden term... and often used to control teachers, schools and students". 
Teachers often see assessment as data for reporting and not for student 
improvement (Carless, 2010). While students see the number grade as the 
main reason for the assessment, an end to the learning process (Winstone & 
Boud, 2020; Winstoneet al., 2017). 

Constructive misalignment 

Many teachers are still struggling with how to design assessments that align 
well with learning activities and outcomes. Assessment drives student learn- 
ing, but teachers in the past have often treated the design of assessment as 
an add-on component to the curriculum design. Given that the majority 
of the teachers are not experts in curriculum design, little thought go into 
assessment design. Teachers sometimes overlook the alignment between the 
assessment approaches and the learning outcomes. For example: using sim- 
ple Multiple Choice Questions (MCQs) to assess students’ understanding of 
a topic; or using essays to measure knowledge or understanding that can be 
assessed using less time-consuming assessment methods. 


iv 


vi 


vii 


viii 
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Overreliance on high stakes examinations 

An examination-oriented system often leads to surface and rote learning 
(Villarroel et al., 2019). The over-reliance on high-stakes examinations or 
traditional summative assessment is a problem of how old habits die hard, 
and many stakeholders often prefer this “rule by numbers” (Kohoutek, 
2014). O’Leary, a chair professor and the Director of the Centre for Assess- 
ment Research Policy and Practice at the Dublin City University said in the 
Irish Times (O’Rourke, 2018), “Parents will think very traditionally” and 
“they will see a test result as more important than teacher’s judgement.” 
Issues of grade distribution, inflation and standards 

Historically, a norm-referencing standard (bell curve) has frequently been 
used, which often disincentivises learning. Adam Grant (2016), professor 
at the Wharton School of the University of Pennsylvania wrote in the New 
York Times, 


The more important argument against grade curves is that they cre- 
ate an atmosphere that’s toxic by pitting students against one another. 
At best, it creates a hypercompetitive culture, and at worst, it sends 
students the message that the world is a zero-sum game: Your success 
means my failure. 


Grant described student collaboration as “a pattern ofcutthroat cooperation”. 
Providing effective feedback to students 

Feedback that is timely and personal can amount to a big workload for 
teachers. Nine years have lapsed since Carless and colleagues (2006a) wrote 
about the seven challenges of assessment, and Carless (2015a) further wrote 
in Times Higher Education that “developing effective feedback in mass 
higher education is difficult: it requires time and careful thought, but classes 
are often large and teachers face multiple demands.” The problem persists. 
The logistics of group assessments 

Group assessment has many benefits for student learning, however, if stu- 
dents are not clear about the objectives and expectations of group work or 
are questioning the validity and fairness of the assessment, it may cause 
confusion and competition within the group. The educational benefit of 
group work will then become less effective, possibly even becoming nega- 
tive. One example would be the use of group assessment to assess students’ 
knowledge when individual assessment can be more effective or efficient. 
Multiple teachers and teaching assistants may also be involved in the mark- 
ing procedures in group assessment, meaning clear criteria and communi- 
cation among the teaching team are absolutely vital. The logistics of group 
assessment is demanding. 

The “unfamiliar” assessment approaches 

Teachers and students alike can be sceptical of innovative or unfamiliar as- 
sessment approaches, including self and peer assessment. Yet, it is the com- 
bination of assessment approaches that can really help students to enhance 
their learning (Lau, 2016). 
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ix Workload pressure from modularisation 
Having highly modular curricula increases students’ assessment workload. 
As an article in the Times Higher Education (McKie, 2018) based on the 
research done by Tomas and Jessop (2019) mentioned, “students are ‘strug- 
gling and juggling’ with assessment loads, lighter assessment loads would 
make room for ‘slow’ and deep learning.” 

x Contract cheating, the latest issue of plagiarism 
The latest issue of plagiarising is not student to student plagiarising or pla- 
giarising from web resources or journal articles but from contract cheating, 
“concerns have been raised about the growing number of websites offering 
students bespoke academic essays in return for a fee” (Price, 2016). 


3.1.4 Assessment Literacy 


As this book focuses on assessment, it will be ignorant not to examine the term 
assessment literacy (AL). Assessment literacy was first formally raised by Stiggins 
(1991). With the advent of outcomes-based learning, which brought about the 
emphasis on student-centred learning, expectations for student learning have 
been made more explicit. The introduction of standards and criterion-referencing 
practices implies that teachers need to understand different dimensions of assess- 
ment from the perspectives of student learning. Public accountability and the 
drive for better educational practice have also emanated the emphasis of AL in 
higher education (Ecclestone, 2001). 

However, despite almost three decades of development, AL is still “clustered 
with competing definitions and models, some of which overlap with related con- 
cepts whose distinctions have not been clearly articulated” (Cowie & Cooper, 
2017; Mandinach & Jimerson, 2016, cited in Medland, 2019; Pastore & 
Andrade, 2019, p. 130). This book will use Paterno’s (cited in Mertler, 2003) 
definition of assessment literacy as the foundation of the concept, as it provides 
a clear and simple interpretation. 


Assessment literacy is the possession of knowledge about the basic principles 
of sound assessment practice, including its terminology, the development 
and use of assessment methodologies and techniques, and familiarity with 
standards of quality in assessment. 

(Paterno, 2001, p. 2 as cited in Mertler, 2003) 


Stakeholders (teachers, administrators, government officials, employers, students 
and parents) need to be assessment literate to use, judge and improve with the 
assessment. 


3.1.4.1 Teacher Assessment Literacy 


According to Stiggins (1995), a teacher is assessment literate if they know what 
they assess, why they assess, how to assess, what the possible problems with 
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assessment are, how to prevent these problems from occurring, and if they are 
familiar with the possible negative consequences of poor, inaccurate assessment. 
And Webb (2002) defined teacher assessment literacy as, 


the knowledge about how to assess what students know and can do, in- 
terpret the results of these assessments, and apply these results to improve 
student learning and program effectiveness. 

(Webb, 2002) 


In recent years, more researchers have been working on comprehensive frame- 
works to conceptualise teacher assessment literacy to create an overall trajectory 
of professional development in this aspect (e.g., Pastore & Andrade, 2019; Xu & 
Brown, 2016). 

Pastore and Andrade (2019) proposed an expanded model of AL that in- 
corporates three overlapping dimensions (i.e. conceptual, praxeological, socio- 
emotional). The conceptual dimension concentrates on the knowledge a teacher 
possesses in terms of the different assessment models and methods. Pastore and 
Andrade provide fundamental questions that lay the foundation of the concep- 
tions of assessment (e.g., purpose, approaches, reporting) such as: 


e What is assessment? 

e Why assess? (e.g. summative, diagnostic, formative, interim, and benchmark); 

e What to assess? (e.g. the content knowledge, skills, thinking, and 
dispositions); 

e How to assess? (e.g. assessment methods, strategies, and instruments and 
how these are relating to validity and reliability). 


The praxeological dimension refers to assessment in practice — their actions. 
This dimension involves how teachers conduct assessments alongside other 
teaching-learning practices to monitor, judge, and manage students’ learning 
outcomes and processes. Examples of these actions include: 


e Designing assessment approaches and ensuring they are constructively 
aligned with the learning outcomes and activities; 

e Designing and applying assessment criteria; 

e Providing evidence of student learning for student improvement and 
certification; 

e Using evidence of student learning to inform curriculum changes and/or 
reforms; 

e Providing constructive feedback; 

e Reporting and communicating assessment results to other stakeholders (e.g. 
parents, other teachers, administrators). 


The socio-emotional dimension concerns ethics, power and relationships. Teach- 
ers who are literate in this dimension would be able to attend to issues such as 
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ethical aspects in assessment, teacher-student power relationship, and emotional 
dynamics of students (e.g., anxiety, frustration). For example: 


e Teamwork interactions relating to assessment impacts and student learning; 

° Minimising unfair grading with clearer guidelines (including self and peer 
assessment); 

° Better design of assessment to reduce unethical academic integrity; 

e Handling students’ reactions to negative feedback and poor assessment re- 
sults well to improve academic resilience. 


Xu and Brown (2016) reconceptualised assessment literacy after reviewing 100 
papers published between 1985 and 2015 from both educational assessment and 
teacher education fields of research, and developed the triangular model titled 
the Teacher Assessment Literacy in Practice (TALiP) framework. The framework 
is based upon personal experiences with teaching and assessment, alongside 
previous models of assessment literacy that emphasised its social nature (Willis 
et al., 2013). The model uses a triangular shape to indicate the knowledge base 
at the bottom, serving as the basis — encompassing factors namely knowledge of 
grading, feedback, peer and self-assessment, ethics in assessment, disciplinary 
knowledge, methods and interpretation. And on the top of the model, there is 
a teacher as an assessor, positing the idea that needs to reflect to (re)construct 
their identity as an assessor. It emphasises the need for teachers to understand 
the varying purposes of assessment and possible assessment strategies that can be 
used to assess student understanding accordingly. 


Teacher Assessment Literacy Awareness Exercise: Are You 
Assessment Literate? 


Based on Pastore and Andrade (2019)'s three dimensions in teacher assessment 
literacy, I have developed the following questions to help teachers deepen their 
awareness, understanding and action in designing and implementing the assess- 
ment. These questions are by no means exhaustive, but it helps teachers to pon- 
der over issues that may occur if they are not fully literate. There are three levels 
(No, Somehow and Yes) in the awareness exercise, by answering the questions, 
readers can evaluate how literate they are in each dimension (Table 3.3). 


3.1.4.2 Student Assessment Literacy 


One of the most recognised works on student assessment literacy is Smith et al.’s 
(2013) research which conceptualised the concept in three dimensions — students 
need to (1) understand the purpose of assessment and how it connects with 
their learning trajectory; (2) be aware of the processes of assessment and how 
they might affect students! capacity to submit the assessment; and (3) be able to 
judge their own work and how to improve. Building on these three dimensions, 
Smith et al. (2013) further developed and validated a student assessment literacy 
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Table 3.3 Teacher assessment literacy awareness exercise 


Teacher Assessment Literacy Awareness Exercise 


Levell Level 2 Level 3 
No Somehow Yes 


The conceptual Do I know why I am assessing? 
dimension - The Do I know what I am assessing? 
understanding Do I know how I am assessing? 

Do I understand the criteria? 

Do I know what standards I am 
employing for the grading? 

Do I know how I should provide 
feedback to achieve the biggest 
impact on student learning? 

Do I know if the assessment is 
valid and reliable? 

Do I know if the assessment 
method complies with the 
institution/faculty/department 
policies? 


Level l | Level 2 Level 3 
No Somehow Yes 


The praxeological Do I know where the best place to 
dimension - The assess? 
action Do I know when I should assess? 

Do I know when I should provide 
grading? 

Do I know when I should provide 
feedback to achieve the biggest 
impact on student learning? 

Do I know if the assessment 
methods address and align 
with all the intended learning 
outcomes? 

Do I know if the assessment 
methods align with the learning 
activities? 

Do I know if the assessment 
methods are actually practical 
and deliverable given the 
available contact time and 
resources (staffing, room 
availability, time)? 

Do I know if the assessment 
methods allow reflection and 
feedback? 

Do I know how I am going to 
report the assessment results? 


(Continued) 
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Levell Level 2 Level 3 


No Somehow Yes 
The socio- Do I know who is the most 
emotional suitable person to grade? 
dimension — Do I know who should provide the 
social and feedback to achieve the biggest 
emotion impact on student learning? 


Do I know the impact of peer 
reaction and remarks? 

Do I know if the assessment 
workload is realistic for teachers 
and students? 

Do I know how the assessment 
results should be reported and 
recognised? 

Do I know if the assessment design 
is rational for students? 

Do I understand the ethics 
and fairness of group work 
assessment? 


instrument to help them evaluate a brief intervention on student learning. The 
above three dimensions have influenced plentiful assessment research (e.g., Car- 
less, 2015b; Charteris & Thomas, 2017; Deeley & Bovill, 2017). 

Research framed exactly under student assessment literacy are scant, and re- 
lated research are mostly investigating how to enhance student assessment liter- 
acy via intervention (e.g., Davari Torshizi & Bahraman, 2019; Deeley & Bovill, 
2017; Smith et al., 2013). There are theoretical studies investigating the con- 
ceptualisation of student assessment literacy (Francis, 2008; O Donovan er al., 
2008), student perceptions of different types of assessment (Flores et al., 2020; 
Pereira et al., 2017; Struyven et al., 2005), and students’ experience in assess- 
ment (O’Donovan, 2019; O*Donovan et al., 2001). A study on students’ evalu- 
ative judgement, that is, the *capability to make decisions about the quality of 
work of self and others? (Tai et al., 2018, p. 467) has also been done. 


Suggestions for Promoting Students’ Assessment Literacy 


Students can become more literate in assessment; the following are some good 
practices in promoting students’ assessment literacy: 


i Spend time to explain course outcomes, activities and assessment; 
ii Share course, programme, department and faculty assessment policies in- 
cluding pass and failure criteria; 
ii Employ various assessment approaches to tackle different outcomes, allow 
students to practice these approaches; 
iv Involve students in the design of the assessment; 
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v Engage students in the discussion of assessment, including the criteria and 
exemplars; 

vi Arrange pre-assessment activities for students to help them get familiar with 
the grade descriptors and assessment approaches; 

vii Introduce self and peer assessment, allowing students to judge their own 
work; 

viii Provide feedback on earlier drafts of the work submitted by students and 
help them focus on the critical comments; 

ix Provide feedback on students’ completed work especially suggestions re- 
garding how to perform better in future tasks; and 

x Include students in board or exam meetings, take their ideas into 
consideration. 


As students become more aware of what and how they are learning, they become 
more motivated. Hence educators need to design assessment for learning, with 
the ultimate aim at designing assessments as learning, rather than merely design- 
ing assessment of learning. 


3.2 Assessment in Experiential Learning 


As mentioned at the beginning of this chapter, one of the major benefits and 
outcomes of experiential learning is the development of holistic competencies, 
yet, the major barrier is the assessment of these competencies. Without proper 
assessment, how would we know what competencies our students need to en- 
hance? In fact, if it is not assessed, how can we convince the teachers and students 
that the development of holistic competencies is important, and that experiential 
learning is value-adding. 

Assessment of holistic competencies is not a simple task. Universities and gov- 
ernments around the world pay special attention to global and regional rankings 
measurement, and given that competencies are heavily emphasized in university 
missions (see Section 1.1) and agendas in government policies, it is surprising 
that there is yet to be any worldwide assessment ranking system of competen- 
cies. There are numerous worldwide university ranking systems, among them, 
the most influential ones are those produced by the Times Higher Education 
(THE), the Quacquarelli Symonds (QS) and the Academic Ranking of World 
Universities (ARWU). The ranking systems include world university, regional, 
world reputation, young university, emerging economies, teaching and impact. 
For research rankings, the criteria used are mostly based on research publications, 
citations, academic performance and faculty expertise including a number of No- 
bel prize winners; and the criteria for teaching rankings, engagement, resources, 
outcomes and facilities are often used as the standards for evaluation. There are 
also different subject discipline rankings, but as of yet, there are no criteria for 
holistic competencies (or 21% century skills) in any of the university rankings. 
In 2015, QS launched the QS Graduate Employability Rankings, however, de- 
spite the title of the rankings, it does not take into account the employability 
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competencies. The criteria are mainly on alumni outcomes, employer-student 
connections, employer reputation, graduate employment rate and partnerships 
with employers. 

Inevitably, there are criticisms on the effectiveness, usefulness and validity of 
these ranking systems (Bekhradnia, 2016; Marope et al., 2013; Marszal, 2012), 
but what I am trying to establish here is that given the importance of compe- 
tencies, global assessment ranking of these competencies does not yet exist as 
assessment of such is daunting. 

Boud (2007) proclaims that: 


Assessment frames what students do. It provides an indication of what the 
institution gives priority to in making judgements, it provides an agenda 
more persuasive than a syllabus or course outline and it therefore has a pow- 
erful backwash effect on all teaching and learning activities. 


Without assessment, both students and teachers would not put the time, effort 
and resources in the development of holistic competencies even if they believe it 
is important. This is echoed by many scholars (Bloxham & Boyd, 2007; Elton & 
Laurillard, 1979; Medland, 2016; Prosser & Trigwell, 1999). 

At the beginning of this chapter, an outcomes-based approach to student learn- 
ing (Ramsden, 2003) was presented, with the three main components (i.e.) learn- 
ing outcomes, teaching and learning activities and assessment shown in Figure 3.1. 
This approach has been widely adopted at all levels of education. However, with 
the importance of reflection and feedback, I feel that it is necessary for these two 
components to be incorporated as part of the outcome-based approach in order for 
different stakeholders to build them into the design of the curriculum. These two 
components are particularly vital in experiential learning development. More often 
than not, feedback is treated as a by-product of assessment and reflection is an un- 
intentional part of the curriculum design. Another important aspect that I would 
like to draw attention to is that, unlike academic knowledge, intended outcomes of 
experiential learning are often unknown. Some of the specific outcomes may not be 
known until during or after the activity (Boud et al., 2020; Luk & Chan, 2021). 
Thus, I have updated the famous outcomes-based approach for experiential learning 
as shown in Figure 3.2. In Chapters 5 and 6, I will elaborate more on reflection and 
feedback in experiential learning. 


3.2.1 The Challenges in Experiential Learning Assessment 


So why is assessment in experiential learning so difficult? Apart from those chal- 
lenges shown in Section 3.1.3 -— the challenges in Academic Knowledge As- 
sessment, the challenges below have been identified specifically for experiential 
learning assessment. 


i Unknown learning outcomes related to experiential learning, or being una- 
ware of them 


activities will help your 


Assessing Academic and Experiential Learning 57 


Learning Outcomes 


What do you want your 
students to learn? 


Teaching and 
Learning Activities 
What types of 


Reflection Feedback 
How can your students reflect What feedback are most 
before, during and after to constructive for 
achieve/enhance the students to enhance 
outcomes? their outcomes? 


Assessment 
How do we know that 
the students have 


rj 
students to learn? gis 


Experiential Learning 
(Other unintended outcomes) 


Figure 3.2 Outcomes-based approach to student learning with reflection and feedback 


ii 


for experiential learning. 


In an article arising from my post-doctorate, Dr. Lillian Luk’s PhD thesis 
(Luk & Chan, 2021), one of the objectives was to identify the learning out- 
comes of internships. Their findings suggested that while there are generic 
types of learning outcomes, there are also specific learning outcomes based 
on each individual student’s experience and motivation. The uncertainty 
surrounding the intended learning outcomes makes it difficult to design 
an effectively aligned assessment. The outcome of experiential learning is 
often not the mastery of disciplinary knowledge, but the acquirement or 
improvement of certain competencies. Critical and careful assessment de- 
sign is needed in ensuring the attainment of expected learning outcomes by 
students (Gosen & Washbush, 2004; Moon, 2004). As explained by Ben- 
nett et al. (2000), “it is usually not until they [generic skills] are included 
in students’ learning objectives and formally assessed, that their importance 
for their future careers is fully accepted by students” (Bennett et al., 2000, 
as cited in Crebert et al., 2004, p. 148). 


Difficulty to design authentic assessments to mimic the real-world 
Students appreciate and become more motivated when assessments are per- 
ceived as authentic. However, designing an authentic assessment takes more 
time and resources. The perceived level of authenticity is also subjective, and 
it varies depending on the individual experience of the students. In a recent 
article, Sotiriadou et al. (2020) tested out a suite of scaffolded authentic 
assessments and oral exams, and the results showed that it improved the 
students’ skills such as professional identity and awareness, and communica- 
tions skills, as well as their employability prospects. When Anna McKie from 
the Times Higher Education (2019) interviewed a number of educational 
experts around the world, they unanimously suggested that “assessment 
needs to be much more closely resembled real-world tasks” to be effective. 
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iv 


vi 


vii 


Lack of clarity about the conceptualisation of competencies 

There is no definition of holistic competencies nor a confined list of com- 
petencies that students are to develop (Chan et al., 2017). In a study by Gu 
(2016), it was found that despite the willingness of teachers in English as a 
Foreign Language education 


to assess intercultural communicative competence (ICC), the EFL 
teachers lack a clear conception of it. This leads to confusion about 
what should be assessed and how to assess it, and to deficiencies in their 
attempts to measure students’ ICC in the classroom. 


No baseline to measure from 

For academic disciplinary knowledge, teachers are aware of students’ prior 
experience and knowledge, as most students are more or less starting at the 
same level before entering university. But with holistic competencies, each 
student has different prior experience and thus, different starting levels of 
competencies. Therefore, it is difficult to measure students’ improvement in 
competencies from the experiential learning activity. 


Hidden curriculum — embedded competencies 

It is quite common to find different competencies embedded within the cur- 
riculum but not explicitly outlined. Assessors find it difficult to assess these 
‘hiddew pieces of the curriculum. 


Assessment misaligns with meaningful activities 

Apart from authenticity, the assessment needs to provide meaning, as many 
ofthe experiential learning activities are meaningful on a personal and emo- 
tional level. Looking at the HCDF framework in Chapter 2, the rationales 
for engaging in the experiential learning activity are very important: teachers 
should be mindful of students’ rationales for joining, was it because of fun, 
or is it career-driven, meaning-driven, because of family reasons or a need 
to obtain credits to complete part of the curriculum? The assessment design 
needs to embrace those and create meaning. Some universities now pro- 
vide experiential learning credits for out-of-classroom learning experiences. 
However, the assessment of these experiences is often a reflective essay of 
300—500 words, and often, no training is provided to either the students 
or the teacher/administrative staff who is responsible for “collecting” the 
essay. Once the reflection essay is collected, the students will automatically 
be given a pass grade. It serves merely as a bureaucratic piece of evidence. 


Unfamiliar assessment approaches in experiential learning 

Experiential learning is unconventional, meaning assessment approaches 
that have evidenced to thrive in one activity may not succeed in another. 
There is no one correct approach to assessing students in experiential learn- 
ing. It may mean that *multiple assessment methods are appropriate", as 
shown in a study with medical students by Roberts et al. (2014). Experien- 
tial education can be seen as a radical change from conventional classroom 
teaching, therefore it is especially important to evaluate its effectiveness 
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through an assessment so that experiential learning, this unconventional 
approach to learning and development, is worthwhile (Voehl, 2018). 


Different levels of Bloom’s taxonomy 

Most teachers are not assessment experts and have very little experience in 
experiential learning. Thus, it is not easy for them to design assessments that 
cover some or all of Bloom’s taxonomy levels. Sometimes, the assessment 
may even discourage meaningful and active learning, especially when there 
are no clear criteria, course content, learning outcomes and assessment. 


Lack of institution support and recognition 

As mentioned previously, if institutions do not take the assessment of ex- 
periential learning seriously, it is difficult for teachers and students to put a 
lot of effort into it, especially when teachers have four pillars of duties (i.e. 
research, teaching, knowledge exchange and administration) to fulfil while 
students have a “packed” timetable. Research (Chan & Lee, 2021; Chan & 
Luk, 2022; Chan et al., 2017) was conducted in the engineering disciplines 
in Hong Kong universities to investigate students’ and teachers’ perceptions 
of holistic competency development. It was found that most students agree 
that holistic competencies are best developed in out-of-class activities or ex- 
tra-curricular curriculum. However, from the interviews with the teachers 
and analysis of the course documents, it was found that a minimal amount 
of activities was organised for students to develop their competencies within 
the curriculum, thus, it was not surprising for students to have the percep- 
tion that competencies are to be best developed in out-of-class activities. 


Not discipline-specific, thus difficult to assess 

The experience developed from experiential learning is often not related to 
the discipline that the students are studying, making it more difficult for 
teachers to assess and relate to the course. Clayton et al. (2003) have found 
from a large dataset of post-secondary education settings that in the absence 
of formal guidelines about how to assess generic skills, [a number of] inform- 
ants [assessors] [...] base their judgements on their general understanding of 
assessment processes, trusting that their training in assessment enables them 
to generate a valid approach. However, one trainer commented that when 
assessing embedded competencies, it was harder to develop assessment tasks 
to ensure coverage, and that more guidance was needed. 


Difficult to provide evidence of student learning 

Quality assurance agencies and funding bodies prefer quantitative data, 
and are looking for value-for-money and value-add, but experiential learn- 
ing is often costly in terms of students’ and staff’s time and perhaps even re- 
sources, thus, it is difficult to provide evidence of student development that 
justify these investments and spendings. If meaningful experiential educa- 
tion is to be achieved, it is essential for universities to capture student devel- 
opment from experiential learning experiences, document and explain the 
data, and finally improve experiential learning activities (Qualters, 2010). 
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xii Logistic issues with many assessors 
As experiential learning involves different parties, the most effective assess- 
ment approach may involve many stakeholders. However, the logistics may 
be difficult (Chan, 2011). 


xiii The who, when and how of feedback 

The idea of who would be the most appropriate person to provide feedback 
in experiential learning education is debatable. Decisions on when and how 
to provide feedback efficiently are also influential. For example, who will be 
the best person to provide appropriate feedback to a student who engages 
in teamwork for a community service project? How to provide constructive 
feedback without discouraging the student in this meaningful project? And 
at the same time, when the feedback does not come across like a criticism, 
will it bring constructive, long-lasting effects? 


xiv Validity and reliability 
Experiential learning assessment approaches need to be creative, but they 
may therefore lack validity and reliability. According to Grimes and Gib- 
bons (2016), "applicability, value and credibility" (p. 109) of assessment is 
of foremost importance. 


xv Assessing the process 
In experiential learning, assessment is often considered part of the learning 
process, so assessment tasks serve as a marking instrument as well as a way 
to enhance and shape learning experiences, especially in helping students to 
make sense of their experiences and connect with previous studies (Moon, 
2004; O"Toole, 2007). This makes an assessment in experiential learning 
particularly essential and beneficial but at the same time, difficult to assess. 


3.2.2 Assessment Literacy in Holistic Competencies 


With competency development being a major outcome in experiential learn- 
ing, the assessment of holistic competencies has been pushed to the front of the 
agenda in many universities worldwide to provide evidence of their development. 
To enhance the effectiveness of assessment, it is important that stakeholders such 
as teachers and students are assessment literate, i.e. obtain “the level of knowl- 
edge, skills, and understanding of assessment principles and practice" (Taylor, 
2009, p. 24), in holistic competency development. Promoting students’ assess- 
ment literacy is to help them comprehend the multiple aspects of assessment, for 
example, the purpose and role of assessment, meaning of the assessment criteria 
and standards, and techniques needed to complete the assessment. This will en- 
able students to recognise and take responsibility for their role in assessment and 
eventually improve their performance. 

In some way, we have only scratched the surface in assessment literacy re- 
search. Assessment literacy research could be found on teacher assessment 
literacy development, but there have been very few studies on students’ AL. Re- 
search based on disciplinary knowledge, such as assessment literacy in language 
learning (Davidson & Coombe, 2018; Fulcher, 2012; Scarino, 2013; Vogt & 
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Tsagari, 2014) and in science (Abell & Siegel, 2011; Gottheiner & Siegel, 2012) 
have also been investigated. Although some conceptual papers took a holistic ap- 
proach to theorise assessment literacy (i.e. conceptualising assessment literacy for 
student learning as a whole instead of for a particular discipline) (e.g., Pastore & 
Andrade, 2019; Xu & Brown, 2016), very few studies to date (Chan & Luk, 
2021; Chan & Luo, 2021a, 2021b) has specifically looked at assessment literacy 
in holistic competencies either within teachers or students. 


3.2.2.1 Student Assessment Literacy in Holistic Competencies 


To address the research gap, together with my research team (Chan & Luo, 
2021a, 2021b), we have explored a student assessment literacy framework in ho- 
listic competency. It represents one of the first attempts to conceptualise student 
assessment literacy in holistic competency development. The study first reviewed 
existing student assessment literacy frameworks, and then explores the extent 
to which these frameworks also apply to assessing holistic competencies based 
on 29 focus group interviews with 122 students from six universities in Hong 
Kong. Findings were subsequently used as a base for discussion towards a new 
framework of student assessment literacy in holistic competency development 
that constitutes four dimensions, i.c. Knowledge, Attitude, Action, and Critique. 
Student assessment literacy in holistic competencies is defined as students’ 
knowledge surrounding assessment in holistic competency development, their atti- 
tude to appreciate and engage in holistic competency development and assessment, 
their action towards assessment tasks and feedback to monitor or further their 
development, and their ability to critique the assessment and feedback provided 
to enhance their holistic competencies. Figure 3.3 visualises the new framework: 
While acknowledging the value of existing student assessment literacy con- 
ceptualisations, Chan and Luo argued that complexities involved in assessing 
holistic competencies require a more nuanced framework. The nebulous nature 
of holistic competencies might have resulted in a lack of standardised guidelines 
on how to assess these competencies (National Centre for Vocational Education 


Knowledge 


Student Assessment Literacy 
in Holistic Competency 
Development 


Figure 3.3 Student assessment literacy in holistic competency development framework 
(Chan & Luo, 2021b). 


Source: Reproduced with permission from authors. 
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Research [NCVER], 2003). “There is considerable potential for invalid judge- 
ments to be made about the quality of learner performance” concerning holistic 
competencies (Clayton et al., 2003, p. 61). Especially for competencies that are 
more personal (e.g., respect, responsibility and honesty), negative assessment 


could be interpreted as a personal affront and lead to ethical issues. 


Taking these features and the interview findings into consideration, the char- 
acteristics of assessment literate students in each of the proposed dimension are 


outlined as follow: 


The Knowledge Dimension 


In the knowledge dimension, assessment literate students 


understand the purposes of assessment for holistic competency devel- 
opment, and the potential side effects assessment may cause; 
understand the processes of holistic competency assessment are not 
always explicit and could be embedded within academic tasks; 
understand holistic competencies can be assessed via different ap- 
proaches and activities, and the connection between assessment ap- 
proaches, activities, and holistic competency development. 


The Attitude Dimension 


In the attitude dimension, assessment literate students 


appreciate holistic competency development in terms of its value and 
show a willingness to engage in its assessment; 

constructively manage the impact of holistic competency assessment on 
emotions and avoid defensiveness when receiving negative feedback; 
possess open-mindedness towards the engagement of holistic compe- 
tency development, assessment and feedback. 


The Action Dimension 


In the action dimension, assessment literate students; 


develop strategies for different assessment tasks; 

reflect intentionally; 

judge and use assessment and feedback for further holistic compe- 
tency development. 


The Critique Dimension 


In the critique dimension, assessment literate students 


recognise they have the right to challenge and critically examine ho- 
listic competency assessment, as well as the feedback provided; 
proactively and critically engage in dialogues with peers, supervisors 
and teachers to improve holistic competency assessment. 
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The framework is expected to bring a number of implications for holistic com- 
petency assessment-related policy, practice and research in higher education. For 
example, the framework can be used as an operationalised model guiding students 
to self-evaluate their engagement in holistic competency assessment and develop- 
ment. Enhanced student assessment literacy will contribute to the improvement 
on holistic competency assessment in practice, enabling more valid and reliable 
certification of these competencies. In the long term, this will also help higher 
education provide concrete evidence of students’ holistic competency development 
for quality assurance units, employers, parents and students themselves. On the 
other hand, the framework further highlights students’ critical engagement and 
understands learners as active agents who exercise discretion in holistic compe- 
tency assessment. Students therefore are no longer perceived as merely “followers” 
of established assessment guidelines, but also as “developers” of these standards. 

Broadly speaking, the assessment landscape in education has experienced dra- 
matic changes in recent decades. The focus has transformed from viewing stu- 
dents at the passive end of assessment, all the way to valuing students’ evaluative 
judgement, i.e. their “capability to make decisions about the quality of work of 
self and others” (Tai et al., 2018, p. 467). Developing students’ and teachers’ 
assessment literacy (see Chan & Luk, 2021 on teacher assessment literacy) in 
holistic competencies is an important move in this trend, which will contrib- 
ute to cultivating a more well-rounded generation. We expect more research 
and practice to advance this important field in the future. Promoting students’ 
assessment literacy is to help them comprehend multiple aspects of assessment, 
for example, the purpose and role of assessment, the meaning of the assessment 
criteria and standards and the techniques needed to complete the assessment. 
This will enable students to recognise and take responsibility for their role in the 
assessment and also improve their performance. 


3.3 The Assessors 


The purposes of the assessment should guide the selection of the type of as- 
sessment employed, and from the type of assessment (i.e. diagnostic, formative, 
summative, learning-oriented assessment or a combination of them), the assess- 
ment approaches (e.g. essay, report, poster, observation, reflective journal) are 
decided. These assessment approaches must also be appropriately aligned with 
the learning outcomes and competencies, and the learning activities. 

However, who the assessors will be is often not deeply considered; tradition- 
ally, the choice has been limited. The tedious yet mandated work of grading is 
often left to the teachers and teaching assistants of the course, as it is difficult to 
request and rely on others to agree to perform this task. In the recent years, this 
has begun to change, as educational researchers and practitioners (e.g. Rolim & 
Isaias, 2018; Sadler & Good, 2006) investigate more diverse ways to assist in 
the reduction of workload and at the same time, take into account students’ ra- 
tionale for learning. University teachers also network with various stakeholders 
to provide diverse environments for student learning. The assessors now include 
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students, peers (junior and senior students), industrial supervisors and experts, 
non-profit organisation coordinators and even the public who are end-users as 
partners for assessing student learning. As an example, at the University of Hong 
Kong (HKU), Chow and Ng (2015) from the Law Faculty invited lay persons as 
simulated clients to assess students’ communicative competencies. HKU Faculty 
of Law is the first law school in Asia to adopt this standardised client programme 
to assess law students’ interviewing skills. 

Experiential learning allows students to be responsible for their learning. 
Many practices have shown that teachers often design experiential learning ac- 
tivities to be student-centred, allowing students to manage the task from the 
beginning to the end, controlling the approach, the deliverables, the process and 
the product. However, when it comes to the assessment, it is back to the gov- 
erning hands of the teachers, and often, it does not align with the activity, is not 
student-centred, adds no meaning to the learning experience, and lacks all the 
elements that are expected in the student experience of the experiential learning 
activity (Ajjawi et al., 2020). The assessment is often an after-thought or a quick 
fix for bureaucratic purposes to satisfy quality assurance and credit-bearing re- 
quirements. In what follows, I present students as partners in assessment from 
three aspects — (i.e.) the benefits, the challenges and some current practices with 
an aim to inform future research and practices on engaging students as partners 
in experiential learning assessment. This is then followed by literature on self, 
peer and group assessment which are more common assessment approaches that 
involve students. 


3.4 Students as Partners in Assessment 


Student partnership or student-staff partnership, which indicates a collabo- 
ration in which students and the teacher work together to further common 
education goals, is particularly complementary to experiential learning. On 
one hand, students as partners are experiential by nature. As pointed out 
by Healey et al. (2010, cited in Healey et al., 2014, p. 36), the central idea 
behind student engagement is “reflecting on the experience of learning by 
doing”. By enabling “a more authentic engagement with the very nature of 
learning itself”, student partnership is often “understood as an experien- 
tial process of reflection and transformation, in relation to oneself and with 
others” (Healey et al., 2014, p. 17). On the other hand, while experiential 
learning does not necessarily involve students as partners, it does help moti- 
vate and prepare students for full partnership by giving learners more inde- 
pendence and encouraging deep engagement (Graham et al., 2007; Healey 
et al., 2014; Trowler, 2010;). That said, although students as partners have 
enjoyed increasing popularity in higher education in recent years, not much 
has taken place in the experiential learning context. Especially in the field 
of assessment, teachers still tend to dominate and govern the entire process 
(Deeley & Bovill, 2017), which undermines the inherent goal of experiential 
learning to engage learners. 
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3.4.1 Benefits in Employing Students as Partners in Experiential 
Learning Assessment 


Student partnership in assessment affords learners and teachers plentiful ben- 
efits. Especially in experiential learning where a range of novel assessment ap- 
proaches are adopted, empowering students in the assessment enables them to 
gain deeper learning experience and acquire multiple competencies such as crit- 
ical evaluation and negotiation (Deeley, 2014; Deeley & Brown, 2014), as well 
as their self-monitoring competencies in order to judge their progress and act 
on gaps in their learning (Carless, 2015b; Evans, 2016). These competencies are 
much sought after by employers and are essential for lifelong learning. 

Although most extant students as partners studies, involving assessment have 
not been based on experiential learning, their findings could be of important 
referential value. One of the most prominent advantages is the enhanced level 
of student assessment literacy, such as a better understanding of the marking 
criteria, the assessment design, giving and receiving feedback and self-/peer- 
assessment. (Deeley & Bovill, 2017; Orsmond et al., 2002; Sambell & Graham, 
2014). Meanwhile, there is ample evidence that student partnership in assess- 
ment also significantly increases students’ motivation and agency in learning, 
helping them to take ownership of and better monitor their work (Abdelmalak, 
2016; Boud & Molloy, 2013; Deeley, 2014; Deeley & Brown, 2014; Fluckiger 
et al., 2010). Evans (2016) highlights the importance of “how students come 
to co-own their programmes with lecturers and see themselves as active con- 
tributors to the assessment feedback process rather than seeing assessment as 
something that is done to them” (p. 2). Through students as partners, students 
are receiving opportunities to be actively engaged in the learning community 
(Deeley & Bovill, 2017; Sambell & Graham, 2010). 

In addition, student partnership in assessment serves to democratise educa- 
tion by creating a co-dependent assessment system that balances the hierarchical 
teacher-student power relationship that often exists (Deeley & Bovill, 2017). 
By advocating a collaborative relationship between students and staff, research 
shows that they can all achieve better mutual understanding (Fluckiger et al., 
2010; Murphy et al., 2017) that may lead to more equal dynamics. Finally, stu- 
dents with first-hand experiences in education can often offer their preferred 
approach and unique insights in improving assessment and feedback. 


3.4.2 Challenges in Promoting Students as Partners in 
Experiential Learning Assessment 


A lack of student partnership in assessment is not an issue exclusive to experien- 
tial learning, but a universal issue in all modes of teaching and learning. Taking 
a broader view, Nash and Winstone (2017) blamed the “consumerist” approach 
to higher education which positions students as passive consumers of service 
and detaches them from their personal responsibilities in learning. Under such a 
consumer model, teachers also tend to believe that teachers are the ones, if not 
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the only ones, to provide quality assessment in an “environment shaped by fees” 
(Murphy et al., 2017). 

Another more entrenched obstacle to promoting students as partners is the 
long-established teacher-student power relationship in assessment as mentioned 
in the benefits section. Traditionally, teachers often exercise full decision-making 
power in the assessment process, whereas students are viewed as the “novice” who 
lack experience and expertise to challenge teachers’ authority (Boud, 2007; Deeley & 
Brown, 2014; Sambell & Graham, 2010).This socially constructed teacher-student 
demarcation is normalised and further strengthened by role expectations in our 
daily practice (Deeley & Bovill, 2017), preventing meaningful partnership from 
taking place in assessment. Under such a power structure, students may find it 
difficult to come up with assessment designs that differ much from what they 
have long been accustomed to (Deeley & Brown, 2014). They could also lack 
confidence (Abdelmalak, 2016) or feel anxious (Deeley & Bovill, 2017) to co- 
design assessment tasks. For teachers, some are concerned that student partnership 
undermines their professional legitimacy as they relinquish power, while others 
entertain doubts as to students’ expertise and maturity in handling the assessment 
in an appropriate manner (Deeley & Brown, 2014; Murphy et al., 2017). 

Students and teachers often have different motivations and expectations for en- 
gaging in partnership, and the differences give rise to tensions around differentials 
in power, reward and recognition of participation, identity, and responsibility for 
partnership work. The partnership between students and teachers is rarely auto- 
matic and requires significant teamwork and consideration on both parts. 

All these issues could potentially lead to resistance to partnership in assess- 
ment and render the students as partners’ practices tokenistic (Cook-Sather, 
Bovill, & Felten, 2014). 


3.4.3 Current Collaborative Assessment Practices in Students as 
Partners 


Healey, Flint, and Harrington (2014) demonstrated four main areas in which 
students often engage as partners through: (a) Learning, teaching and assess- 
ment; (b) Subject-based research and inquiry; (c) Scholarship of teaching and 
learning; and (d) Curriculum design and pedagogic advice and consultancy. In 
general, there should not be any boundaries as to what students should get in- 
volved as co-partners, if the idea is to have a partnership. Partnership is defined 
as a “reciprocal process through which all participants have the opportunity to 
contribute equally, although not necessarily in the same ways, to curricular or 
pedagogical conceptualisation, decision-making, implementation, investigation, 
or analysis” (Cook-Sather et al., 2014, pp. 6-7) 

Students commonly engage themselves as partners in four main areas of 
assessment — designing assessment, marking criteria, self and peer assessment. 
So far, extant research has mostly been conducted in Australia, Europe or North 
America, focusing on students’ perceptions of their partnership experience in 
assessment (Giles et al., 2004). To name a few examples, Deeley (2014) introduced 
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a co-assessment approach for her service learning course where students were re- 
quired to give oral presentations to demonstrate employability skills and attributes 
developed through service learning. In addition to the teacher’s assessment, each 
student also self-assessed their oral presentation. Subsequently, the teacher had 
individual appointments with each student to discuss their presentation, the dis- 
crepancy in assessment, and to agree on a mark. In another of Deeley’s students as 
partners study (Deeley & Bovill, 2017), though not based in an experiential learn- 
ing context, students were further engaged in co-designing their essay titles and 
marking criteria, which they subsequently used to self and peer assessment in the 
public policy courses. Likewise, Abdelmalak's (2016) research also demonstrates 
how six mature students collaboratively designed course assignments and grading 
standards and decided how they would be assessed in an education course. 

However, the above studies all set out to investigate students’ perceptions, 
which resulted in a range of mixed outcomes as challenges in assessment. To 
enrich students as partners! practices, future rescarch in assessment to better 
inform education practices, such as the agreement between marking from dif- 
ferent sources (e.g., Orsmond et al., 2002) and evidence of students’ learning 
improvement should be carried out. 


3.4.4 How to Involve Students as Partners in Assessment? 


There are many areas in which students can get involved in the assessment. Be- 
low are some suggestions divided into five different categories (Table 3.4). I have 
just submitted a systematic review paper on student partnership in higher educa- 
tion assessment, I expect that would be published by the end of 2022. You may 
find more information on how to involve your students in this upcoming paper. 

To truly embrace students as partners in assessment, teachers need to treat the 
partnership equally, to overcome the relinquishing of power they so highly hold. 


3.5 Self-Assessment 


Self-assessment is an assessment method that allows students to assess their own 
performance, as it is self-reliant, it helps students to be autonomous learners. It 
can be extremely valuable in helping students develop self-reflection, criticality 
and judgment and ultimately, learn how to be responsible for their own learn- 
ing. The drawback lies in the fact that students may not be familiar with this 
self-motivated assessment, and may lead to over- or under-evaluation of their 
performance given how self-assessment may be subjective. Self-assessment is 
particularly effective for experiential learning education because students can 
be more aware of their strengths and weaknesses and understand themselves. 
As student experience may vary daily in experiential learning, this type of self- 
assessment helps students rework reflection (i.e.) providing a *second reflection 
action" opportunity (Chan & Wong, 2021)? Self-assessments are more often 
used as part of a formative assessment process, rather than a summative one. 
In Adachi et al. (2018) study, academics shared their belief in self-assessment: 
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Table 3.4 Five categories involving students as partners in assessment 


5 Categories Involving Students as Partners in Assessment 


Policy establishment * Collect student input in institutional assessment protocols 
* Invite student members to develop assessment policy and 
regulations 
* Involve students in the establishment of assessment appeal 
guidelines 
Involve students in plagiarism policy and regulations 
Evaluation and quality Invite students to sit on quality assurance panels in 
assurance programmes, university and national levels 
* Involve students in the complete accreditation process 
with professional bodies 
° Co-design course/institutional evaluation surveys with 
students 
Assessment and feedback Partner with students to negotiate the assessment 
design approaches and timeline 
° Co-develop assessment criteria and rubrics 
e Co-design the assessment by gathering students’ inputs or 
involving them in the actual design 
* Design the feedback mechanism with students on 
questions such as who, how and when to provide the most 
constructive feedback 
* Explore possibilities on partnership in designing 
innovative assessment 
* Allow flexibility for students to generate their learning 
outcomes and aligned assessment in experiential learning 
(Luk & Chan, 2021) 
Marking and providing Give student opportunities to self and peer assess 
feedback * Involve students in making decisions on group assessment 
and issues with free-riders 
* Allow students to critique the feedback provided 
° Set up student-led feedback mechanisms to help each 
other (e.g. Gilley & Clarkston, 2014) 
Assessment literacy * Host joint professional development on assessment 
literacy for both students and teachers 
° Discuss students’ and teacher's rights and responsibilities 
° Organise regular meetings with students to discuss issues 
in assessment and feedback 


“you know as lifelong learners the ability to be able to self-assess, to judge where 
you're at. And say, ‘Look, I think Pm under here, or I think Im performing well 
here and why?". I think that is a really key transferable skill that our students 
need to learn' and students become critical independent learners. 


3.6 Peer-Assessment 


Peer Assessment is an assessment that allows students to assess each other's per- 
formance. Students reflect on others’ performance in both academic knowledge 
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and holistic competencies. It can be extremely valuable in helping students 
learn from each other through listening, analysing and problem solving. For 
example, in my own presentation assessment class, students often reflect the 
do’s and don’ts from watching each other presenting. Students are then asked 
to provide one strength, one weakness and one area for improvement for the 
student presenter, and this gives students the opportunity to be exposed to 
different perspectives, critique and judge, and ultimately, learn how to be re- 
sponsible for their own learning. Critique and judging are skills in themselves; 
students need to learn how to critique and not criticise, and provide fair and 
unbiased judgement. This is also a great way to provide and receive more feed- 
back from others and reduce teachers’ workload. Similar to self-assessment, 
there is the issue of unfamiliarity with such assessment; students may not be 
able to judge the levels of performance, and with peer assessment, sometimes 
students may feel like they are under peer pressure. Adachi, Tai, and Dawson 
(2018) has shown that with peer assessment, students develop the ability to 
learn collaboratively, as it involves judging others’ performance, and they also 
understand the criteria standards better and develop feedback and other trans- 
ferable skills. 


3.7 Group Assessment 


Group work is a form of cooperative learning. Teamwork, communication, and 
project management skills are known competencies that are activated while en- 
gaging in group work (Burke, 2011), and as industries often require teams to 
work together, group work helps to prepare students for their future careers. As 
members of a team, students feel a sense of responsibility, and that can ideally 
motivate their efforts and contributions to the project. They also learn different 
perspectives through collaborations with their peers. However, when assessing 
groups, clear criteria and guidelines on the process and product are vital to en- 
sure valid and fair grading for both collective and individual contributions. It is 
relatively easy to grade the product of group work, but more difficult to grade 
the process. With reference to Chan (2010), examples of criteria which may be 
appropriate for assessing the process of group work include: 


° Demonstrate leadership; 

e Attitude of being cooperative; 

e Application of a variety of methods to deal with different tasks and 
difficulties; 

e Engagement and motivation with the assigned task; 

e Ability to motivate other people; 

e Willingness to listen and responsiveness to feedback and opinions. 


In Chapters 4 and 8, cases on how teachers are assessing in experiential learning 
will be shown. 
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Conclusions 


In this chapter, assessment in academic knowledge and experiential learning 
were examined in detail. Readers learnt about the challenges in both domains 
and how they apply these concepts in experiential learning. By understanding 
the challenges, teachers, designers, administrators, senior managers and others 
can put on their thinking hats to design effective assessments for experiential 
learning and avoid some of these known issues. Good assessment design can mo- 
tivate students and stakeholders, particularly, teachers and students need to be 
assessment literate to become more motivated. The Teacher Assessment Literacy 
Awareness Exercise provides simple questions for teachers, teaching assistants 
and those who are going through the design process. It is not just a literacy 
exercise; it can also be used as an assessment design guideline for both academic 
knowledge and experiential learning. 

Assessment literacy for students is equally important. Our 4-dimensional 
Student Assessment Literacy for Holistic Competencies framework contains an 
important message, that is, student needs to be able to critique, to develop, to 
judge and to self-assess the feedback, the assessment and the activity. This brings 
out the connected relationship between students as partners and experiential 
learning education. 

In the next chapter, we will present common assessment approaches for the 
different types of experiential learning. 


Questions to Ponder 


e Are you assessment literate? Rate yourself using the Teacher Assess- 
ment Literacy Awareness Exercise based on a 3-Likert scale; 

e How will you involve your students as partners in assessment? Use 
the five Categories Involving Students as Partners in Assessment as a 
checklist. 


Personal Reflection 


Ilike to borrow William Shakespeare's quote in Hamlet and slightly mod- 
ified it to 

“To assess or not to assess... that is the question!” 

Assessment drives learning and can capture your students’ attention, but 
for experiential learning, when the activity itself is so meaningful, do we 
want to ruin it by assessing it? How can we ensure that students continue 
to achieve impact, develop and not focus on grades or credits alone? 
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Notes 


1 Holistic competency is defined by the author as an umbrella term for generic skills 
(e.g. teamwork, self-management, creativity), attitudes (e.g. resilience, hardworking) 
and virtues (e.g. respect, honesty, dignity). 

2 Rework reflection is defined as the type of reflection approach which allows students 
the opportunity to perform ‘at least’ a ‘second reflection action’, allowing the stu- 
dents to rework on their reflection from their initial reflection. The rework reflection 
may be from their own reflection or from reflecting on peers’ reflection. This can be a 
very powerful reflective approach if teachers provide a suitable environment. Students 
can become responsible for their own learning or in reflective terminologies, reflexiv- 
ity in reflection. Cited directly from Chan and Wong (2021). 
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4 Designing Experiential 
Learning Assessment 


Assessment in the future may not be the kind of assessment that you and I tra- 
ditional grew up with. The word assessment may mean something completely 
different. Or new term may need to coin for the type of assessment deems suitable 
for experiential learning. 

— Chan, CKY 


Introduction 


This chapter begins by providing a set of (A-D) essential questions for teachers 
and educational developers to ponder when designing experiential learning as- 
sessments. It demonstrates via a genuine case study on how these questions can 
be used to assist teachers in designing the assessment properly. The chapter then 
examines diverse assessment approaches that are commonly used within the con- 
text of experiential learning, such as portfolios, reflective journals, concept maps, 
participation, marking sheets, and observation, and discusses different types of 
experiential learning activities such as community service learning, field trip, 
internship, student exchanges, project competition, etc. 


4.1 Essential Questions for Effective Assessment Design 
in Experiential Learning 


In Chapter 3, the foundation of various assessment terminologies, such as diag- 
nostic, summative, formative and learning-oriented assessments was presented 
and a detailed account of the kind of challenges in academic knowledge and in 
experiential learning assessment was explored to prepare the readers on how to 
effectively design experiential learning assessment. 

Effective assessment design needs to take into account many of the afore- 
mentioned challenges, such as the absence of clear learning outcomes and the 
confounding variables, in Chapter 3 (Ewert & Sibthorp, 2009). Qualters (2010) 
acknowledged that effective assessment approaches in experiential learning must 
be able to “separate perceived learning from genuine learning” and at the same 
time, accurately report the learning process and students’ development. When 
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designing assessments, the priority is to identify their purposes. Qualters (2010) 
posed four essential questions for teachers to seriously consider before they de- 
sign assessments for experiential learning (adapted from Elman [1993]). These 
are (1) WHY are we doing the assessment? (2) WHAT are we assessing? (3) HOW 
do we want to assess in the broadest terms? (4) HOW will the results be used? 

These questions are important, but I felt that Qualters’ questions are too 
broad and do not provide insights into the design. Given experiential learning 
activity requires students to be self-responsible and take ownership of their learn- 
ing experience, Qualters’ questions are not able to accurately guide teachers in 
assessment design. Grounded on the elements from the HCDF (Chan & Yeung, 
2020) and the mirrors from the Chain of Mirrors as discussed in Chapter 2, I 
recommend using the following questions to guide the assessment plan in a step- 
by-step approach. There are four main questions known as (A-D), then each 
main question is followed by a subset of questions to allow further in-depth con- 
templation for the assessment design. Table 4.1 shows Chan's (A-D) Essential 
Questions for Effective Assessment Design in Experiential Learning. 


4.1.1 Sample Case Study — Assessment Design for Oversea Student 
Exchange Programme Using Chan’s (A-D) Essential Questions 
for Effective Assessment Design in Experiential Learning 


At a university in Ireland, engineering students in their third year can opt to spend 
a semester in an oversea university (for example, in Germany, France, the United 
States or Hong Kong) as an exchange student. Students undertake respective courses 
that are mapped to their current curriculum, and may also opt to take some electives 
that are of interest to them in the oversea university. The credits achieved from the 
courses are transferred back to their home university and form part of their Grade 
Point Average (GPA). The assessments used aligned with the learning outcomes 
of the courses and are usually conventional assessments within most engineering 
courses, targeting mainly academic knowledge in the engineering discipline. 

In order to not risk their GPA, most students do not choose the student ex- 
change opportunity (note that this aligns with the Holistic Competency Devel- 
opment Framework (HCDE), it shows that student can lack motives to join and 
experience the activity. For the student exchange, students avoid the experience, 
that is their approach to NOT develop). Furthermore, student exchange organ- 
izers and teachers tend to encourage only those students who can achieve high 
grades to participate. They also discourage students to take electives that are not 
relevant to the chosen field of engineering, such as common core or language 
courses, while they are overseas. 

Looking at the assessment, are the assessments effectively designed for an 
oversea student exchange programme? 

Applying Chan’s (A-D), let's look at Question A, why are we assessing? We 
are assessing because the students need the credits to proceed to the next level 
and also, we need to provide evidence that student exchange is a value-added 
experience for our students. 
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Table 4.1 Chan’s (A—D) essential questions for effective assessment design in 
experiential learning 


A WHY am I assessing? 


l Isit for quality assurance, summative, formative, diagnostic, enhancing 
learning or a combination of these purposes? 
2 Isitactually an acceptable reason to assess? 


B WHAT learning outcomes should I be assessing? 


l Academic knowledge and/or holistic competencies? 

2 Are these learning outcomes “hidden or known" within the activity? 

3 Are these learning outcomes directly related to student's rationales for 
engaging in the experiential learning activity? [According to HCDF (chapter 2), 
if student’s rationale is fulfilled, then the student will engage in the activity and 
the assessment. ] 

4 Do these learning outcomes involve student's inputs? 

5 Do I need to assess ALL the learning outcomes? 


C HOW should I assess these learning outcomes? 
1 What assessment approaches should I employ? 


i Why am I selecting this assessment approach? 

H Is this approach able to explicitly measure the learning outcomes? 

iii Does this approach embrace student's rationales for engaging in the 
experiential learning activity? [According to HCDF, if student's rationale is 
fulfilled, then the student will engage in the activity and the assessment.] 

iv Does this approach make sense for the students given the activity 
experienced? 

v Does this approach allow for reflection?" 

vi What does excellent and poor assessment look like in this approach? 

vii Is it actually useful, feasible and meaningful for the students and teachers 
to be assessed using this assessment? 

viii Who should be involved in the assessment? 

ix When should the assessment take place? 


2 How many assessment approaches do I need to measure each learning 
outcome? [Back to C.(1) if there are many more assessment approaches] 


D Using these assessments, WHAT evidence and feedback will I provide to the 
different stakeholders? 


l For students, WHAT kind, by WHOM and WHEN should I provide the 
feedback? 

2 For teachers, WHAT kind, by WHOM and WHEN should I provide the 
feedback? 

3 For University, WHAT should I provide as the evidence for quality assurance? 

4 For Funding bodies, WHAT should I provide as the evidence for quality 
assurance? 


a Avery important question in experiential learning. 


Consider Question B, what learning outcomes should I be assessing? As men- 
tioned, the actual learning outcomes are those related to academic knowledge, 
but are they the appropriate learning outcomes for an oversea student exchange 
programme? It seems that the programme has not considered other hidden 
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learning outcomes or students’ actual rationales for their participation in an 
oversea student exchange programme. Put it simply, if you were an exchange stu- 
dent, why would you participate in a student exchange programme, what would 
you anticipate to learn, to do and to develop? According to Montrose (2015), 
“many study abroad programmes simply transfer academic credits from one tra- 
ditional discipline-based educational institution to another without intention- 
ally utilising the international experience itself as the basis for learning” (p. 7). 

As mentioned, rationale strongly influences student’s participation and ap- 
proach to develop; it has been evidenced to be an important element in the HCDF 
and Chain of Mirrors. The expectations of an oversea student exchange are to 
experience life outside their home country and take the opportunities to make 
new friends, learn and immerse into a different culture. Yet, holistic competencies 
such as global citizenship, adaptability and confidence are not part of the learning 
outcomes, and without the learning outcomes, the relevant assessment also does 
not exist. This culminates to be one of the biggest issues with experiential learn- 
ing assessment design. Despite those learning outcomes being the jewels of the 
oversea student exchange programme, they are hidden and not assessed. As many 
students end up being assessed only on their discipline knowledge in the exchange 
programme, they choose to concentrate on building up their academic knowl- 
edge to achieve the high grades required for their GPA out of time-management 
concerns. Possible reflection on the exchange experience and learning gains from 
residing in an oversea country are significantly neglected. Many students may not 
even make new friends. Without being assessed on it, they are less engaged in their 
experience, and their approach to developing those competencies is discounted. 

Some may argue that students may find time to immerse more in the culture with- 
out being assessed on those intangible learning outcomes, but the opportunity to 
provide direct evidence to universities and funding bodies on students’ value-added 
experience will be missed. And from the example above, some learners may neglect 
valuable experiences in student exchange, given only the assessment of academic 
knowledge forms part ofthe GPA. Thus, the key is to balance the answer to question 
(C) (1.e.) to check if the assessment useful, feasible and meaningful for students and 
teachers. Grimes and Gibbons (2016) stated that, 


if there are no definitive or absolutes on how we assess, we must seek the 
means by which to establish and ensure the applicability, value and credibil- 
ity of the relevant process if we wish to justify and monitor what is taking 
place. 


(p. 109) 


In this example, Questions C and D become meaningless for these students 
given that the steps defined by the first two questions were not designed appro- 
priately. The genuine learning outcomes are misplaced, and the assessment does 
not align with the learning experience. 

In Chapter 8, many excellent cases will be presented, I invite you to use the 
Chan's (A-D) questions and apply them to the cases in order for you to under- 
stand how assessment can be effectively designed. 
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4.2 Assessing Holistic Competencies in Experiential 
Learning Activities 


As some holistic competencies are better developed inside the classroom and 
some outside, it would not be feasible to assess all types of competencies in one 
activity (Clanchy & Ballard, 1995; Crebert et al., 2004). Therefore, universities 
should provide diverse learning opportunities for students to develop different 
holistic competencies to become well-rounded individuals. To achieve this goal, 
it is important to expose our students to various learning experiences throughout 
their studies, such as in-class experiences where students develop competencies 
alongside disciplinary knowledge and out-of-class experiences where students 
enjoy a variety of other authentic and enriching learning opportunities. In the 
figure below (this figure first appears in Chapter 2, for ease of reading, the figure 
is presented here again), I present a classification of possible learning opportu- 
nities for students’ holistic competencies development. As shown in the figure, 
competencies can be developed in many activities, in fact, students can develop 
competencies during non-academic related opportunities such as participation 
in scouts or sports, but these opportunities are often not assessed or recorded. 


Classification of learning activities and opportunities for holistic competencies 
development 


E.g. community service learning”; field 
trips*; internships*; work placements”; 
student exchange’; student teaching*; 


Out-of-class 


Discipline- E.g. capstone projects*; problem-based 
specific learning (PBL) *; presentations*; clinical 
and non-clinical practicums*; laboratory*; 
In-class research fellowship*; group projects*; 
; workshops* 
Academic 
related E.g. common core courses*; residential 
education; camps; general education*; 
student societies and clubs; workshops 
Non- Extra- and guest speakers; sports activities; 
discipline curricular career guidance; entrepreneurship-related 
specific activities/competition; externship; 


community service learning; field trips; 
internships; work placements; 


INon-dendenmie elaed E.g. experiences in daily life; work 


experiences; interactions with friends 
and family; charity and non-profit 
organisation participation; media, 
internet and socialisation 


* Maybe credit bearing. 


Figure 4.1 Classification of the types of learning activities (in-class, out-of-class and 
extra-curricular) for the development of holistic competency in university 
(Duplicate here for ease of reading). 
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4.3 Experiential Learning Activities and the Common 
Assessment Approaches 


Deployment of assessment approaches depends on the purposes, intended 
learning outcomes, type of field-based experiential learning in which the stu- 
dents will participate, and many other factors. As shown above, there are 
many different types of activities, with even more emerging, that teachers are 
innovatively designing every day. There is no one-hat fits all in assessment, 
thus, in the section below, effective assessment approaches that have been 
frequently employed in common types of field-based experiential learning ac- 
tivities are presented. In Chapter 8, we will provide some best assessment 
practices used around the world that are tailor-made for specific experiential 
learning activities, these assessment practices may give us a glimpse on how 
teacher may design our own effective innovative assessment in experiential 
learning. 


4.3.1 Common Assessment Approaches 


In this section, some of the common assessment approaches in experiential 
learning are presented in detail. These common approaches can be used to as- 
sess both academic knowledge and holistic competencies. As this book focuses 
on experiential learning, the contexts of these common assessment approaches 
are intended for learning outcomes that arise from common experiential learn- 
ing activities. I have employed a presentation format for these assessment ap- 
proaches so that readers can easily read through the approach of their interest. 
Each approach begins with an introduction and the structure of the approach, 
then a list of advantages and disadvantages follow, and after, a list to show how 
the assessment can be designed effectively, finally, it ends with a sample rubric 
for readers to employ and adapt. Common assessment approaches presented 
include: 


Blogs Portfolios Reflective Journal 
Direct Observation Posters Short Answer Questions 
Learning contracts Presentation Written Reports 


In addition to the approaches discussed below, readers can also find sample 
rubrics at the end of the chapter for 


Logbooks Peer-assessment Self-assessment Video project assessment Worksheets 
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Blog is short for web log. It is an asynchronous platform for posting messages 
and materials for discussions and sharing. Blogs add a participatory dimension 
to their assessment through facilitation of participatory culture. According 
to researchers (Barwell et al., 2011; Gardner & Ladyshewsky, 2008; Hung & 
Huang, 2016; Williams & Jacobs, 2004), in comparison to the conventional 
teacher-centered approach, blog facilitates a student-centered approach to learn- 
ing and assessment; the learning is driven by students through their sharing, 
comments, reflections and feedback. The interactive nature of blogs ‘brings ... 
learners’ thoughts and arguments into the open and allows for discussion and 
restructuring of knowledge to occur’ (Gardner & Ladyshewsky, 2008, p. 242). 
Through inter-blogging discussion, collaborative production and peer review 
(Allen, 1999; Halic et al., 2010; O'Neill et al., 2011), students are given the 
opportunity to learn, assess, and receive an evaluation from their peers and im- 
prove their evaluative judgement. These foster students’ development of holistic 
competencies like communication skills and self-reflection, while upholding the 
spirit of experiential learning at the same time (Fessakis et al., 2008; Gardner & 
Ladyshewsky, 2008). Together with technology, it can be a favourable approach 
for students in the digital age (Garcia et al., 2019). 


Structure of Blogs 


The structure of blogs is simple. Depending on the platform used, it commonly 
allows images, videos, podcasts and embedded links in the post and has a sec- 
tion where readers can leave comments at the end of the post. It may also allow 
private and public postings depending on the privacy settings. A blog can be an 
individual or group work, and each entry can have one author or many. Blog 
entries are typically displayed in reverse chronological order with the most recent 
post displayed as the highest post on the page. 


Take time to set 


Take time to answer 


Take time to correct 


Take time to provide feedback 


Suitable for a large class 


KIKI KKK 


Can use technology 


Passive approach 


Y Active approach 
Y Process Oriented Method 
Product Oriented Method 

P = Possibly Y =Yes 
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Advantages of Blog Assessment 


1 


DA 


Wu 


New opportunities for learning about cooperation and idea exchange (espe- 
cially when class time is limited). 

Fostering learner autonomy. 

Facilitating higher levels of learning and competencies such as critical thinking. 
Providing opportunities for students to enhance IT and other digital liter- 
acy skills. 

Students feel safer and participate in a more organic way. 

Strengthening and expanding the learning community, reaching wider 
audiences. 

Convenience and flexibility in terms of time and space. 

Allows self and peer assessment. 


Disadvantages of Blog Assessment 


1 


Partial and superficial participation: Participation level varies largely among 
participants and a few students may dominate the discussion. Students may 
not take it as seriously as face-to-face discussion. It requires a longer period 
of time to assess. 

Unstructured feedback: Students may or may not get responses or feedback 
immediately because not all entries and responses happen at the same time. 
Unfamiliar assessment and feedback approach: Blog assessment is still new 
to both students and teachers, so its marking criteria are not always clear. 
Similarly, promoting self and peer feedback may be foreign to many students 
(Caldwell & Heaton, 2016). 

Distraction: Students are more likely to go off topics and get distracted. It is 
difficult to ensure learning outcomes are met. 

Time-consuming: Student assessors and teachers need more time to review. 


How to Design an Effective Blog Assessment? 


1 


It is important that instructions and expectations of blog assessment are 
clearly communicated to students in advance. Ensure students know what 
the objectives of the blog assessment are. 

Provide assessors with examples of how to provide constructive feedback. 
Provide students the time period, location, guidelines, requirements and 
assessment criteria. Students should also be aware of who is going to assess 
them - teacher, tutor, peers and/or self? And if peer- and/or self-assessment 
are employed, decide if the weightings would be the same as the teacher’s 
assessment. 

Prepare a structured marking sheet for all assessors, so all assessors are fa- 
miliar with the assessment criteria. Do you grade every entry? Or do you 
grade selected posts? Do you provide marks for feedback and comments? Is 
it the frequency of posts that you mark or the quality of posts? For example, 
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Levin and Davis’ (2007) assessment of a blog task for a marketing promo- 
tions course was based on the frequency of postings. Students needed to 
post at least nine times to obtain a maximum of 30 points. 
5 Itis fine if blog assessment is not associated with marks, teachers may pre- 


fer to use blog assessments as a formative assessment with no grading (Ta- 


ble 4.2 shows a sample marking rubric for assessing students’ blogs during 
their exchange studies). 


Blog — Marking Rubrics 


Table 4.2 A sample rubric for assessing students’ blogs during their exchange studies 


Marking Rubrics Excellent Proficient Average Poor 
Grading 
Descriptors 
Tdeas and - posts provide - posts generally - posts record - posts appear 
contents an interesting, provide an the student’s irrelevant to 
(intercultural critical and interesting, exchange life the student’s 
awareness) comprehensive critical and and some are exchange life, 
record of comprehensive presented in an or are presented 
the student’s record of interesting and poorly, showing 
exchange life the student’s critical manner little to no self- 
which shows exchange life with minimal reflection from 
a high level of which shows self-reflection the student. 
self-reflection some level of in intercultural 
in intercultural self-reflection communication. 
communication. in intercultural 
communication. 
Frequency - updates posts - updates posts as - updates posts as  - does not 
more frequently ` required. required, but at update posts 
than required times needs to as required; 
and keeps a be reminded. the posting 
regular posting frequency is 
record. unsatisfactory. 
Visual - uses high - uses mostly - very little - does not use 
enhancement quality high-quality evidence of multimedia to 


multimedia to 
promote the 
visual appeal 
and readability 
of the blog. 


Direct Observation 


multimedia to 
promote the 
visual appeal and 
readability of the 
blog; sometimes 


the usage may not 


be appropriate. 


multimedia 
usage to 
promote the 
visual appeal 
and readability 
of the blog. 


promote the 
visual appeal 
and readability 
of the blog; 
readability of 
contents is 
weak. 


Direct Observation assessment is exactly as the name suggested — the asses- 
sors observe the students’ performance and assess their level of ability based 
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on their performance. The Council on Social Work Education refers to this 
type of observation as “observations in real time” (Council on Social Work 
Education, 2015, as cited in Dill, 2018). Clinical-related disciplines often use 
a form of direct observation known as Objective Structured Clinical Exam- 
ination (OSCE) to assess students’ practical skills. Teachers may sometimes 
use direct observation to judge each student’s input in group work activities 
to prevent free-riders. In experiential learning, direct observation can provide 
an accurate account of not just how the students perform, but observers can 
also evaluate many of the holistic competencies that students develop during 
the course of the experiential learning activity. Observation assessment is 
only effective when it follows a systematic plan to help both the assessor and 
the student focus on what need to be observed and recorded. An oral assess- 
ment is often used as a follow-up assessment to supplement observations of 
students’ performance. According to Beddoe et al. (2011), observations are 
more than just ‘watching’, as those who are observing are also being ob- 
served, learners learn by observing the observers and that can lead to emerg- 
ing competence. Observation can be subjective, and teachers and researchers 
(Gauthier, 2019; Maxwell, 2001) are sceptical about how deep direct obser- 
vation may be possible, as they feel that this assessment approach may not 
easily assess higher-order learning outcomes (e.g. critical thinking), thus, it 
is often supplemented by other forms of assessment. However, for holistic 
competencies such as teamwork, sometimes, there is no effective alternative 
to direct observation. 


Structure of Direct Observation Assessment 


The structure of an observation assessment greatly depends on the disci- 
pline in which the assessment takes place, and it also depends on whether 
the assessor is observing the entire activity or only part of it. In general, the 
assessor will observe for five to ten minutes, make a note to help with the 
feedback and grading, and sometimes follow up with an interview or oral 
assessment. 


Take time to set 


Take time to answer 


Take time to correct 


KK TK IK 


Take time to provide feedback 


Suitable for a large class 


Can use technology 


Passive 

X Active 

Y Process Oriented Method 
Y Product-Oriented Method 
P- Possibly Y ses 
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Advantages of Direct Observation 


1 


2 
3 


Observation may sometimes be the only assessment method possible that 
accurately assess a learning outcome. 

There can be no plagiarism or false reports. 

It is a great way to assess practical skills and holistic competencies. 


Disadvantages of Direct Observation 


1 


Some studies have shown that direct observation may not assess higher- 
order levels of learning outcomes (Beard et al., 2009), and teachers often 
felt that it is not adequate for a full assessment. Thus, other assessment ap- 
proaches may be required. However, other researchers have found that it has 
been used to assess higher-order levels of learning outcomes such as critical 
thinking (Brunt, 2005; Paul, 2014). 

Direct observation assessment requires a lot of time to assess and to prepare, 
thus, it is an expensive way of assessing. 

The presence of the observer can change a student’s performance as being 
watched can be intimidating for many students. Furthermore, the dynamics 
of the observation room may change as the observer/assessor enters. It is 
often debatable whether the observer/assessor should be visible or hidden 
(Kazdin, 1982; Labaree, 2002). So where, who and how the observation is 
being assessed are factors which may affect a good observation assessment. 

There is no student anonymity in direct observation (Coffelt, 2017). 

To ensure high efficiency and reliability, clear grading standards for all par- 
ties are essential. However, grading criteria for observation assessment can 
be difficult to design, develop and follow. 

Immediate feedback is useful but is sometimes difficult due to time 
constraints. 

Practical work is usually ephemeral and dissenting views may later be con- 
tested if notes or recordings are not documented clearly. 

Direct observation can be subjective. 


How to Design an Effective Direct Observation Assessment? 


Ensure students know what the objectives of the observation assessment are. 
Provide students the time period, location, guidelines, requirements and 
assessment criteria. Students should also be aware of who is going to assess 
them - teacher, tutor, peers and/or self? And if peer- or self-assessment are em- 
ployed, decide if the weightings would be the same as the teacher's assessment. 
Prepare a structured marking sheet for all assessors, so all assessors are fa- 
miliar with the assessment criteria. 

Feedback is very important for a good observation assessment (Table 4.3 
shows a sample rubric for a direct observation assessment that can be used in 
a community service project partnered with a non-profit organisation. This 
is the rubrics used by the community service partner to assess the student's 
performance within a group). 
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Learning Contract 


A learning contract, also known as negotiated learning agreement, is a formal 
written agreement used when planning for a learning project or activity. Accord- 
ing to Knowles (1986) who pioneered the use of learning contracts as a mech- 
anism for self-directed learning, learning contracts serve as “an alternate way of 
structuring a learning experience.” 

In some learning programmes, learning contracts are merely agreed on with 
no assessment carried out. This practice stems from the belief that the process of 
planning and conducting a contract itself is constructive without being assessed 
(Powers, as cited in Boak, 1998). However, the assessment of the contracted 
learning process is said to be essential in motivating student learning and evalu- 
ating the effectiveness of the contract (Boak, 1998). Commonly and most effec- 
tively, throughout the programme or activity, students would arrange meetings 
with advisors or teachers to review progress and share ideas (Knowles, 1986). 
Advisors are also the ones who would evaluate students! work as evidenced in 
the learning contracts and give the final grade (usually pass/fail, depending on 
the course nature and requirements). 

In some courses, the learning contract is implemented like a traditional assign- 
ment which would be assessed and evaluated based on the specific criteria. Pos- 
sible criteria include how well the learning objectives listed on the contract met 
course expectations; or how relevant the learning strategies are to the learning 
objectives. However, commonly, only the learning process and accomplishment- 
type learning objectives are evaluated (Knowles, 1986). 

Negotiation and commitment by both the learner and the advisor are essen- 
tial elements in contract learning (Anderson et al., 2013). Negotiated through 
discussions, both parties seek consensus on how particular activities should 
be conducted to achieve specific learning objectives, and the relevant roles 
and responsibilities they each have to fulfil (Anderson et al., 2013; Knowles, 
1986). The acquirement of holistic competency can be listed as a type of 
learning objectives in learning contracts. Before drafting the learning con- 
tract, the advisor should assist the learner in locating and establishing their 
learning needs relevant to the course expectations and develop realistic learn- 
ing objectives accordingly (Anderson et al., 2013). In some cases, the con- 
tract learning processes would also include tasks for learners to ponder their 
learning needs, or in other words, their weaknesses related to the programme 
objectives. 

Ongoing consultation is likely necessary to review learning progress and mo- 
tivate the learner (Knowles, 1986). Hence, we can see that although it might 
not be explicitly presented, the analysis of individual learning needs always 
takes place in contract learning, often during negotiations between advisors 
and learners. 

To develop successful learning contracts, students reflect upon their learn- 
ing and arrange discussions with teachers or advisors, while the latter offer on- 
going support and advice, keep track with the direction and the relevance of 
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the project, balance students’ learning needs and course requirements, as well 
as maintain the educational standard (Anderson et al., 2013). Throughout the 
negotiation process, teachers or advisors and students develop a collaborative 
relationship with open communication and mutual respect. 


Structure of Learning Contract 


The structure of a learning contract can vary depending on its purposes. How- 
ever, according to Anderson et al. (2013), a learning contract usually consists of 
the following sections: 


e The learning objectives of the learning project; 

e The resources and strategies for achieving the learning objectives; 

e The evidence produced for indicating the achievement of the learning 
objectives; 

e The timeline and completion date; 

e The assessment criteria applied for assessing the evidence. 


Take time to set 
Take time to answer 


Y 
Y 
Y Take time to correct 

X Take time to provide feedback 
Y 

Y 


Suitable for large class 


Can use technology 


Passive 

Y Active 

Y Process-Oriented Method 

Product Oriented Method 
P-Possibl Y ses 


Apart from the five typical sections, it may also include the area of knowledge 
and the specific competency that the learning contract addresses. For some con- 
tracts, there could even be a separate section allowing advisors to write com- 
ments or students to record important notes during discussions. 

Contract learning is a very flexible learning method. Course teachers can 
adapt learning contracts into arguably any discipline and any course. How- 
ever, not all contracts contain all the above-mentioned elements, and not all 
contracts grant total freedom to students. In some contracts, learning objec- 
tives are prescribed by the instructor and learners can only choose their own 
learning strategies; some contracts would provide predetermined options for 
students to choose from (for example, a list of course objectives which students 
can amend, add to, or omit); other contracts require input from learners re- 
garding the learning objectives (ranges from total freedom to minimal input; 
Knowles, 1986). 
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Advantages of Learning Contract 


According to Anderson et al. (2013), learning contracts allow 


1 Individual differences to be acknowledged. Learners are allowed to tailor- 
make learning activities that suit their specific needs and according to their 
own interests. Hence, students! engagement and motivation are usually en- 
hanced when compared to traditional learning. 

2 Students to have more autonomy in their learning. More autonomy, in turn, 
means more responsibility to bear. Students take a more active role in their 
learning, developing their self-directed learning potential. 

3 Students to plan the development of their holistic competencies. 

4 Students to reflect on their learning. Students gain a better understanding 
of the nature of learning in which they are engaged. It also allows students 
to identify their most preferred learning approach. Students are more inde- 
pendent in their learning and rely less on their teachers and advisors. 

5 Students to be able to develop a more democratic relationship with their 
teachers/advisors, compared to the traditional teacher-student relationship. 
The parties become learning partners. 


Disadvantages of Learning Contract 


According to Anderson et al. (2013), disadvantages of learning contracts include 


1 Not all learners are capable to plan and write learning contracts as these 
abilities Oe, planning and writing) are not innate but learned (Brookfield, 
1985). 

2 New learners may feel stressed when initially engaging with learning con- 
tracts because of the unconventional flexibility. Further guidance is required 
when introducing learning contracts to new learners because the subsequent 
effectiveness will be affected. 

3 Learners may be uncomfortable with learning contracts as they are often 
not the best judge of what is expected to learn and to what extent. 

4 The drafting of a learning contract requires students to have a certain un- 
derstanding of the subject (at least an overview of the topic) in advance, 
especially if this subject is new to the learner. Without sufficient background 
knowledge, the effectiveness of the learning contract is in doubt. 

5 Learning contracts may not be appropriate for courses that are highly 
informational. 

6 The negotiation process of learning contracts between advisors and students 
is very time-consuming, especially if this is done for the first time. 

7 When assigning learners to advisors, there may be very little choice or de- 
sign involved. If there are disagreements, especially during one-to-one ne- 
gotiation and without further backing and support, there can be a concern 
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for learner vulnerability and power dynamics. In addition, if the subject 
expertise of the assigned advisor does not match learners’ needs, the level of 
guidance possible will be limited. 

Learning contract is still relatively new in higher education. Both teach- 
ers and students are potentially anxious and reluctant to adopt this new 
method, particularly if this is used as an assessment. 


How to Design an Effective Learning Contract Assessment? 


According to Anderson et al. (2013), to design an effective learning contract 
assessment 


Students and teachers or advisors engage in discussions at the initial stage 
to identify a learning need with relevance to the course requirement, before 
proceeding to establish specific objectives. Teachers/Advisors assist students 
(the learners) to pinpoint what they want to learn, could learn and should 
learn (Cross, 1992), which also clarifies to students what is negotiable and 
what is not. 

Teachers or advisors assist students to develop specific objectives by refining 
the learning need to be identified. Teachers or advisors should ensure that 
students have the capability to achieve the objectives developed under the 
given time and resources, while simultaneously, making sure the objectives 
reflect the perceived needs of students (the learners). 

Teachers/Advisors should assist students in identifying the best resources 
available for achieving the objectives, and in deciding on the best strategies 
to obtain required information and materials. 

Students and teachers/advisors need to discuss and reach a clear consensus 
on what is to be assessed, as well as what evidence, aligning with each spe- 
cific learning objective, is to be provided as proof of achievement. 

Students and teachers/advisors need to negotiate and determine the assess- 
ment criteria that are the most appropriate and well-aligned with various 
learning objectives, depending on their nature. An effective learning con- 
tract assessment requires both parties to reach a consensus on the appropri- 
ate quality standards of the completed project. 

While students are allowed to negotiate with the teachers/advisors on the 
assessment criteria, well-defined minimum criteria should be provided to 
students for reference. Relevant conventions (e.g. word length, form, pro- 
cedures for referencing, etc.), depending on the type of project, should be 
clarified to students. 

Obtaining feedback from relevant experts is important in learning contracts. 
Teachers/Advisors should discuss with students how the feedback provided 
can be incorporated into the assessment. 

Teachers/Advisors should also negotiate with students regarding the form 
of the assessment (the form of feedback to be provided, the need for an 
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evaluation report, if marks or grades are to be used to indicate achievement 
level, etc.) tailored to suit the best needs of students. 

9 The effectiveness of the completed learning contract is determined by how 
well the assessment criteria are met. A completed learning contract is sel- 
dom developed from one single consultation with the teacher/advisor. Stu- 
dents need to carefully review the draft of the learning contract proposal. 
(Table 4.4 shows a sample marking rubric for assessing student's learning 
contract). 


Application Example of Learning Contract — The HAVE U Can 
Programme in Hong Kong 


In the summer of 2018, students from six universities in Hong Kong were invited 
to join a non-credit five-day residential programme known as the HAVE U Can 
programme (Chan, 2019), to enhance their holistic competency, particularly in 
creativity, critical thinking, appreciation, consideration, responsibility, leadership, 
teamwork, resilience and communication. About 200 first- and second-year un- 
dergraduates from various disciplines enrolled in the programme. Students were 
given the opportunity to stay in residential colleges at the University of Hong 
Kong and were put into small groups of five or six. Each group was mentored 
by two trained senior student mentors. A mentoring kit was provided to each 
pair of the student mentors with ice-breaking questions and games as well as 
FAQs, programme information, and other safety guidelines. The participating 
students must first attend an individual face-to-face interview. At the interview, 
the students were given information about the programme, and were asked about 
their rationale for joining the programme. All students were interviewed by the 
teaching assistants who were fresh graduates, and the interviews were conducted 
in a casual and relaxed manner. 

The programme is a non-credit bearing programme that occurred in the sum- 
mer, it was important to ensure that the students selected would be committed 
to the programme. By interviewing the students, it would allow us to select 
those students who were genuinely motivated and introduced elements of the 
programme that would engage students to develop their holistic competency. 
This was the idea behind the Rationale/ Motives mirror in the Chain of Mirrors 
model presented in Chapter 2. The information of the programme also provided 
the foundation for the Expectation mirror. 

On the first day of the programme, students met with their groups and their 
senior student mentors. Students had to fill in a learning contract and a vali- 
dated self-awareness survey on their holistic competency (Chapter 9 shows a 
sample of a learning contract; For the validated survey, check out [Chan & Luk, 
2020]). This helped students understand themselves better and commit to their 
team and the programme. These activities are grounded by the Expectations and 
Self-Awareness mirrors in the Chain of Mirrors model. 

For more information on learning contracts, please read Anderson et al. 
(2013). Learning contracts. A practical guide. Routledge. 
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Portfolio 


A portfolio is a collection of a student’s work which provides evidence showing how 
the student can meet the specified learning outcomes. A typical portfolio consists 
of work selected by the student, reasons for selecting these works and self-reflection 
on the learning process. A portfolio reflects students’ developmental process; thus, 
the student or teacher does not only assess the product, but also the learning pro- 
cess in which the student engages during the given period. Portfolio is an assess- 
ment method that monitors the growth and development of student learning. 


Structure of Portfolio Assessment 


Unlike most assessments, portfolio assessment can contain many different forms 
of assessments, as it is a collection of a student’s work. A portfolio assessment is 
sometimes followed by an oral assessment or presentation. 


There are three types of assessment portfolios 


e Documentation Portfolio is used to highlight the development and improve- 
ment of student learning during a given period of time. It often contains a 
range of artefacts from brainstormed lists to rough drafts to finished products; 

° Process Portfolio is similar to a documentation portfolio, in which it contains 
all the evidences required to prove the learning outcomes in the given time, 
in addition, it integrates reflection and higher-order cognitive activities. It em- 
phasizes metacognitive functioning and encourages students to become active 
participants in understanding their own learning. Process portfolio often con- 
tains documentation of reflection such as learning logs, journals and diaries; 

° Product Portfolio is a portfolio demonstrating a student's best work. This type 
of portfolio is typically used for the interview. It is more of a summative assess- 
ment and typically does not include students’ reflections on the learning process. 


Take time to set 


Take time to answer 


Take time to correct 


Take time to provide feedback 


Suitable for a large class 


KLK IKK IK 


Can use technology 


Passive 

Y Active 

Y Process-Oriented Method 
Y Product-Oriented Method 
P-Possibl Y -Yes 


Advantages of Portfolio 


1 Portfolio is an assessment method that gives students the opportunity to be 
responsible for their own learning. Students often develop proud ownership 
of their work. 
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2 


CON WD OD d 


Self-reflection in each step allows students to improve as they see themselves 
progressing over time at the different stages. 

Portfolio is an authentic assessment method, it involves meaningful work 
and often has personal relevance. 

It promotes diversity of assessment methods. 

It motivates through the visibility of the final portfolio. 

It promotes creativity, individuality and uniqueness in the assessment of learning. 
It assesses all levels of Bloom’s taxonomy. 

It shifts teachers’ focus from comparative ranking to improving understand- 
ing via feedback. 

Learning should not be all about the end result, portfolio is one of those 
assessment methods which allows students to demonstrate more than the 
end result — a process-oriented method. 


10 High validity. 


Disadvantages of Portfolio 


1 


2 


Portfolio is a very time-consuming assessment method in terms of planning, 
preparing, correcting and providing feedback. 

Clear instructions and guidelines must be given to students, as students 
often take portfolios solely as a collection of their work with no justification 
of reasoning and reflective statements. A sample or defined portfolio size 
should also be given. 

Plagiarism can occur. 

It is a subjective assessment method and teachers sometimes find it difficult 
to assess, as it is difficult to ensure reliability. 

Students may overspend their time on the presentation of the portfolio, and 
not on the actual content. Tutors and peer assessors may also place too much 
focus on the aesthetic aspects of the portfolio and overlook the meaningful 
ideas behind the topic. 

As portfolio is a process-oriented assessment approach, for maximum bene- 
fits, it should span across a number of semesters. However, it is unusual for 
students to have the same teacher over a number of semesters, and without 
that continuity, it is difficult for teachers to assess. 


How to Design a Good Portfolio Assessment 


iS) 


Ensure the students know what the objectives of the portfolio assessment are. 
An exemplar and defined portfolio size should be provided. 

Provide students the time period, location, guidelines, requirements and assess- 
ment criteria. Students should also be aware of who is going to assess them — 
teacher, tutor, peers and/or self? And if peer- and/or self-assessment are em- 
ployed, decide ifthe weightings would be the same as the teacher’s assessment. 
Prepare a structured marking sheet for all assessors, so all assessors are fa- 
miliar with the assessment criteria. 

Feedback design should be included in the assessment design (Table 4.5 shows 
a sample marking rubric to assess students’ portfolio for field trip study.). 
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Poster 


Poster assessment is one of the many ways of presenting the content and findings 
on a topic to an audience or a group of audiences through condensed infor- 
mation, often including pictures and graphs. It is often used to assess student 
learning in group research projects. Peer, self and tutor assessments can be used 
as part of the grading process. 


Structure of Posters 


Poster assessment usually involves students researching a topic and presenting 
their findings on a poster. Although question and answer sessions are uncom- 
mon, students are sometimes requested to stand by their posters to explain their 
findings. Poster assessments are expected to be brief and attractive. Online plat- 
forms (such as Miro, InVision, Concept board) allow posters to be presented 
digitally so they can be assessed and commented by many assessors and peers at 
different times. 


A good poster is usually expected to have the following two characteristics: 


e Good contents; 
e Good and clear visuals. 


'Take time to set 


Y 'Take time to answer 


Y 'Take time to correct 


Take time to provide feedback 


x Suitable for a large class 

Y Can use technology 
Passive 

Y Active 


Process-Oriented Method 
Y Product-Oriented Method 
P = Possibly Y ses 


Advantages of Posters 


l Poster assessment encourages creativity. 

2 Poster assessment is short and succinct. This would require the students 
to think distinctively and select the important elements that need to be on 
display. The ability to summarise is important. 

3 Poster assessment can be assessed by peers at different times even without 
the presence of the creator. 
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Disadvantages of Posters 


1 


It is important for the assessors to state the assessment criteria explicitly; 
the students need to know if the content of the material is part of the 
criteria and/or the method of presenting the poster is part of the criteria. 
If students are to be assessed on different aspects (such as creativity skills 
or presentation skills on the posters) other than the content, they should 
be given the opportunity to learn and practice those aspects before being 
assessed. 

Students may overspend their time on the visual effects, and not on the 
actual content. Tutors and peer assessors may also focus on the aes- 
thetic aspects of the poster and overlook the meaningful ideas behind 
the topic. 


How to Design a Good Poster Assessment? 


1 


Ensure the students know what the primary objective of the poster assess- 
ment is. 

Let the students know if they are required to be around for poster 
explanation. 

Students should have training on how to summarise and select the impor- 
tant parts to display in a poster. 

Provide students the time period, location, guidelines, requirements and 
assessment criteria. Students should also be aware of who is going to assess 
them - teacher, tutor, peers and/or self? And if peer- and/or self-assessment 
are employed, decide if the weightings would be the same as the teacher’s 
assessment. 

Be specific with the assessment criteria, and prepare a structured marking 
sheet for all assessors so that all assessors are familiar with the assessment 
criteria. 

Peer feedback is an effective way for students to receive prompt feedback 
in poster assessment (Table 4.6 shows a sample rubric assessing students’ 
posters and is suitable for a student-faculty research partnership project (e.g. 
undergraduateresearch projects). 
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106 Designing Experiential Learning Assessment 
Presentation 


Presentation is the process of showing and explaining the content of a topic to an 
audience or a group of audiences orally. It is often used to assess student learning 
in individual or group projects. In recent times, presentation is no longer just 
about oral presentation but also the visuals. Paper, white board or PowerPoint 
presentations are sample tools to aid the visual part of the presentation. Peer, 
community partners, industrial and academic supervisors can all be assessors, 
this would allow the collection of feedback from different stakeholders if the 
topic or presentation style generates subjective opinions or different views. 

Students may be assessed in terms of content relation, knowledge grasp, 
presentation style, enthusiasm, and audience engagement. It is important that 
teachers provide clear criteria on what will be assessed for the presentation, so 
students can spend time focusing on the expected outcomes. 


Structure of Presentation Assessment 


Presentation assessment usually consists of a topic on which the student research, 
discusses and presents. Question and answer sessions usually follows the pres- 
entation. This measures the ability of students to respond critically, think under 
pressure and manage discussions. Sometimes it is in this part of the presentation 
that a student shows his or her in-depth knowledge of the topic and presentation 
skills. A good presentation is usually expected to consist of 


e An introduction, rationale and aims 

e Literature on which the study is based upon 

e Major points and ideas explained and summarized 

e  Results/Related points/Issues/or others depending on the topic 
e Conclusion - future work 

e The presentation should be carried out within the time limit 


Take time to set 


Y Take time to answer 

Y Take time to correct 

Y Take time to provide feedback 
Suitable for a large class 

Y Can use technology 
Passive 

Y Active 


Y Process- Oriented Method 
Y Product-Oriented Method 
P-Possib Y =Yes @ 


Advantages of Presentation Assessment 


1 Humans tend to remember actions and behaviours easier than words 
through reading, writing and listening. Observing other peers presenting 
will help students reflect on oneself and avoid repeating others’ mistakes. 


Designing Experiential Learning Assessment 107 


Presentation is an effective method for improving students’ public speaking 
skills which can be very useful for their careers. 

Presentation is often part of the overall assessment for a project. It allows stu- 
dents to provide a detailed summary of the project to the assessors, allowing 
the assessors to ask questions and probe more deeply into students’ report. 
Presentation provides an effective way to assess competencies such as pres- 
entation, communication, information literacy, organisation, creativity and 
critical thinking competencies. 


Disadvantages of Presentation Assessment 


1 


Presentation does not take a long time to mark but it does take a relative 
amount of time for the students to present during contact hours, usually 
making it not the best method for a large class. 

It is important for the assessors to state the assessment criteria explicitly. The 
students need to know if the content of the material is part of the criteria 
and/or the method of presenting is part of the criteria. If students are to be 
assessed on different aspects other than the content, they should be given the 
opportunity to learn about and practice those aspects before being assessed. 
If the skills needed in a live presentation are not part of the intended learn- 
ing outcomes, the presentation may not be a suitable assessment method. 
Students may overspend their time on flashy animation, software and other 
high-tech sound effects, and not on the actual knowledge contents. Tu- 
tors and peer assessors may place too much focus on the aesthetic aspects 
of students’ presentations and overlook the meaningful ideas behind the 
topic. 

Presentations may not be very inclusive. Students with special learning needs 
or physically impairment may struggle. 


How to Design an Effective Presentation Assessment? 


1 


Ensure the students know what the primary objective of the presentation 
assessment is. 

Tell them how long the presentation will be, the strictness of the timekeep- 
ing and whether there is time for Q&A. 

Let students know the assessment criteria and marking scheme, the students 
should also be aware of who is going to assess them, for example, tutor, 
peers and/or self? And if peers or themselves are going to assess, would the 
weightings be the same as the tutor’s assessment? 

Prepare a structured marking sheet for all assessors, so all assessors are fa- 
miliar with the assessment criteria. 

Feedback is very important for an effective presentation assessment to sup- 
port student’s development and is best to be provided immediately after the 
presentation (Table 4.7 shows a sample rubric for a presentation assessment 
used in a capstone project. This is the rubrics used by the peers to assess the 
student’s performance.). 
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Reflective Journal 


A reflective journal is a means of recording ideas, personal thoughts and expe- 
riences, as well as reflections and insights a student gained during the process 
of learning in a course. Reflective journal requires the students to think more 
deeply, to challenge their old ideas with new incoming information, to synthe- 
size the course materials they have learnt into their personal thoughts and phi- 
losophy, and also to integrate it into their daily experiences and future actions. 
The benefits of the reflective learning process usually accumulate over a period of 
time, during which the students usually show a series of developmental changes, 
personal growth and changes in perspectives. 


Structure of Reflective Journal 


Basically, there are two standard forms of reflective journals: 


e Structured journals: students are given a specific question, target, or set of 
guidelines to base their writings on. 

e Unstructured journals/free-form journals: students are required to record 
thoughts and feeling with minimal direction. 


However, both structured journals and unstructured journals should contain 
some of the following aspects: 


e To discuss or argue a journal paper/report/an issue from a private stand- 
point or from various perspectives. 

e To synthesize or analyse some materials or resources for building up an 
argument. 

e To examine a particular perspective or issue with prior or new knowledge. 

e To generate questions and think deeply after examining relevant resources 

e To describe personal experiences and integrate them into the issues in 
concern. 

e To express freely for or against specific questions given by teachers/peers/ 
supervisors. 

e To develop the ability and critical attitude to integrate learning into real- 
world experiences. 


Although reflective journals are often presented in a written format, they do 
not necessarily have to be in written format. Some reflective journals can be 
audio and video recorded or presented online (Chan & Wong, 2021, more in 
Chapter 5). The presentation format of reflective journals is flexible. Reflective 
journal is suggested to be applicable to a broad range of disciplines. 
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Take time to set 


Take time to answer 


Take time to correct 


Take time to provide feedback 


Suitable for a large class 


KIKI KIKI K 


Can use technology 


Passive learning 


< 


Active learning 

Process-Oriented Method 

Product-Oriented Method 
P = Possibly Y =Yes 


< |< 


Advantages of Reflective Journal 


1 


Active learning — The process of reflection encourages students to take the 
initiative to be active, self-driven; allows the individual learner to explore 
concepts and ideas in relation to their thoughts and feelings from different 
perspectives. Students can become independent thinkers through practice 
and enable themselves to solve various problems on their own. 
Understanding the progress of students — Reflective journals provide good 
opportunities for teachers to gain a better understanding of how their stu- 
dents think and feel about the course, and the learning progress of their 
students throughout the course, which will eventually help them enhance 
their students’ learning. 

Improving writing skills — Writing reflective journals can involve students in 
a new form of writing which they may not have had a chance to experience in 
the past. This exposure can bring out improvement in students’ writing skills. 
Freely expressing personal views and criticising of one-self — Reflective jour- 
nal assignments provide the platform for students to freely express what they 
think and feel about the course and their learning process, and also promote 
their expression of ideas, personal experiences and opinions. This is an ideal 
place for students who are generally not willing to speak up in the class and 
tutorials to express themselves. 

Enhance critical thinking and creativity — The process of self-reflection en- 
hances the development of critical thinking skills among students when they 
relate their knowledge to real-world issues. It can help students develop their 
creativity and a questioning attitude towards different issues and problems. 


Disadvantages of Reflective Journal 


1 


Difficult for objective marking — Due to the subjective nature of reflective 
assignments, it is rather difficult for assessors to be objective and have con- 
sistent grading. Different assessors may have quite a large discrepancy in 
their judgment of the same reflective piece. 
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Time consuming for grading — The context of reflective writing can often be 
very diverse, and involves a wide range of concepts, issues, and perspectives. 
As a result, it often takes considerable amount of time for assessors to read 
and grade students’ works. 

Confidentiality — As students have to disclose their personal and private 
views and information in their reflection, some of them may be unwilling to 
honestly disclose their perspectives. They may be concerned that what they 
write will significantly affect the grade they receive. 

Clear guidelines needed — Many students may not be familiar with the re- 
flective writing genre and may feel very lost when working on it for the first 
time. Teachers have to give clear guidelines to students about what should 
be included in the reflective journals, what can be learnt from writing it, as 
well as how they will be graded. 


How to Design a Good Reflective Journal Assessment? 


1 


un 


Consider the types of reflective journals that fit your course (if students 
are inexperienced with reflective journals, the structured form with specific 
questions and guidelines available should be more “student-friendly”). 
Make sure clear expectations and assessment criteria are given to the stu- 
dents. (e.g. What can students put in their journals? What is the defi- 
nition of “reflection”? What is the approximate length for each journal 
entry?). 

Try to make students understand the purpose and benefits of reflective jour- 
nals at the very beginning. 

Make sure that teachers have explained and discussed the policies concern- 
ing privacy and confidentiality of information with students. 

Decide the regularity of journal entry (e.g. weekly, monthly). 

Provide timely feedback to students (Table 4.8 shows a sample mark- 
ing rubric to assess students’ reflective journals on their service learning 
experience.). 
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Short Answer Questions 


Short-answer questions are open-ended questions that require students to re- 
spond with an answer. They are commonly used in examinations or quizzes to 
assess the basic knowledge and understanding (low cognitive levels) of a topic, 
prefacing more in-depth assessment questions. It can be used in a field-trip for 
students to answer quick questions. 


Structure of Short Answer Questions 


Short-answer questions do not have a generic structure. Questions may require stu- 
dents to respond by completing the sentence, supplying the missing word, writing 
short descriptive or qualitative answers, explain diagrams etc. The answer is usually 


short, from one word to a few lines. Students may often answer in bullet form. 


Take time to set 


Take time to answer 


Take time to correct 


Take time to provide feedback 


Y Suitable for large class 
Y Can use technology 
Y Passive 

Active 


Process Oriented Method 
X. Product Oriented Method 
P-Possibly Y -Yes 


Example: 


MHz measures the of the computer. 

List the different types of plastic surgery procedures. 

Choose the kind of flowers that you can find in the national park from the 
list below. 


Advantages of Short Answer Questions 


1 


2 


Short-answer questions are relatively fast to mark and can be marked by 
different assessors. 

Short-answer questions are also relatively easy to set compared to other as- 
sessment methods. 

Short-answer questions can be used as part of a formative and summative assess- 
ment, as the structure of short answer questions are very similar to examination 
questions, students are more familiar with the format and feel less anxious. 
Unlike MCQs, students must supply an answer and no options are provided 
for guessing. 
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Disadvantages of Short Answer Questions 


1 


Short-answer questions are only suitable for questions that can be answered 
with short responses. It is crucial for the assessor to be very clear about the 
type of answers expected when setting the questions, because short-answer 
questions are often open-ended questions, so students are free to answer any 
way they choose. Short-answer questions can lead to difficulties in grading 
if the question is not worded carefully. 

Short-answer questions are typically used for assessing knowledge only, stu- 
dents may often memorize answers for these questions by rote learning. Ifas- 
sessors wish to use short-answer questions to assess deeper learning, careful 
attention (and frequent practice) on appropriate question design is required. 
Accuracy of assessment may be influenced by handwriting /spelling skills. 
There can be time management issues when answering short-answer 
questions. 


How to Design a Good Short Answer Question? 


1 


CNA 0 


10 


11 


12 


Design short answer items that can assess the learning objectives 
appropriately. 

Express the questions with clear wordings and language that are appropriate 
to the student population. 

Ensure there is at least one correct answer for each question. 

Ensure that the item clearly specifies how the question should be answered 
(e.g. Student should answer it briefly and concisely using a single word or short 
phrase? Are students given a specific number of blanks to answer the question?) 
Consider whether the positioning of the item promote efficient scoring. 
Write the instructions clearly so as to specify the desired response. 

Set the questions explicitly and precisely. 

Direct questions are better than those which require completing the 
sentences. 

For numerical answers, let the students know if they will receive marks for 
showing partial work (process-based) or only the results (product-based), 
also indicate the importance of the units. 

Let the students know what your marking style is like. For example, is bullet 
point format acceptable? Or do students have to respond in full sentence 
format? 

Prepare a structured marking sheet; allocate marks or partial marks for ac- 
ceptable answer(s). 

Be prepared to accept other equally acceptable answers, some of which you 
may not have predicted (Table 4.9 shows a marking rubric to assess students’ 
short answer questions related to their capstone project. Short answer ques- 
tions tend to be short, and have more precise answers, thus, it is possible for 
each question to list out all the possible answers/point). 
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Short Answer Questions - Marking Rubrics 


Table 4.9 A rubric to assess students’ short answer questions related to their capstone 


project 
Marking Excellent 
Rubrics 
Grading 
Descriptors 


Proficient 


Average 


Poor 


Completeness ^ the responses 


Master of 
knowledge 


Knowledge 
transfer 


Writing 
mechanics 


are complete 
and all parts/ 
questions are 
addressed to 
the fullest 
extent. 


- the answers 


are accurate, 
detailed, 

and show 

the students’ 
advanced 
understandings 
of the academic 
knowledge. 


- demonstrates 


effective 
application 
of academic 
knowledge to 
the project. 


- language use 


is mature, free 
from errors. 


Written report 


- the responses 


are complete; 
however not 
every part/ 
question is 
addressed to 
the fullest 
extent. 


- the answers 


are mostly 
accurate, 
detailed and 
show the 
students’ 
adequate 
understandings 
of the academic 
knowledge; 


- the answers 


support the 
student’s 
satisfactory 
learning 
outcomes. 


- demonstrates 


some level of 
application 
of academic 
knowledge to 
the project. 


- language use 


is appropriate 
with minor 
errors. 


- the responses 


are mostly 
complete; only 
some parts/ 
questions 

are fully 
addressed. 


- the answers 


should be 
more accurate 
and detailed; 
answers show 
the students’ 
limited 
understandings 
of the academic 
knowledge; 


- the answers to 


some extent 
support the 
student’s 
learning 
outcomes. 


- demonstrates 


insufficient 
application 
of academic 
knowledge to 
the project. 


- improvement is 


needed as there 
are apparent 
errors. 


- the responses 


are not 
complete; 
many parts/ 
questions 
are not fully 
addressed 
or even 
addressed. 


- the answers 


are not 
accurate or 
detailed, 
which show 

a lack of 
understanding 
of the 
academic 
knowledge; 


- the answers 


reveal the 
student’s 
unsatisfactory 
learning 
outcomes. 


- there is a lack 


of application 
of academic 
knowledge to 
the project. 


- language use 


is problematic 
with errors. 


Reports are documents that present detailed interpretation and content, and 
critical analysis of the results of an experiment, investigation and project on 
a particular topic. A report usually contains different sections, such as intro- 
duction, methods, results, discussion and conclusion. There may be a specific 
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writing style and format for the report that vary depending on the discipline 
and activity. There are various types of reports across different subjects, such as 
analytical reports or business proposal in the business field, and practical labo- 
ratory reports in some science subjects. Report writing is an important skill for 
students, as it is often used in all work fields. 


Structure of Written Report 


Take time to set 


Y Take time to answer 

Y Take time to correct 

X Take time to provide feedback g 

Y Suitable for large class E 

Y Can use technology S, 

Y Passive E. 
Active š 


Process Oriented Method 
Y Product Oriented Method 
P = Possibly Y =Yes 


In general, a report is made up of different sections with specific information. 


° Cover page: Provide basic information such as course details, title of the 
report and personal information. 

e Abstract: A brief summary (sometimes known as executive summary) of the 
report which is approximately 100-150 words. It includes the aim of the 
report, a brief research methodology used, summary of key findings, a short 
discussion, and conclusion. 

e Introduction: Contains the objective, rationale and background informa- 
tion of the research/experiment/project. It may cover some aspects such 
as the relevant theoretical perspectives, a brief review on previous and ex- 
isting knowledge, limitations of previous work, hypotheses and expected 
outcomes. 

e Method: Outlines the approaches and procedures that were used to carry 
out the research/experiment/project. This section also lists the specific ma- 
terials and equipment used for the experiment/research. In some scientific 
reports, instructions of the experiments and some labelled diagrams are in- 
cluded to give more information of the practical work. 

e Results: Describes the findings and observations. Data can be presented in 
tables, graphs and even through calculation. In some scientific reports, the 
results section also shows if the hypothesis of a theory is supported or not. 
This section is a summary of the key findings. However, it would not cover 
any explanation of the findings and observations. 

e Discussion: Provides the interpretations and evaluations of the findings and 
Observations; analyses the findings in relation to the theoretical background 
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and objective of the study; comments on any unexpected outcomes; ac- 
knowledges any problems and limitations with possible reasons; gives sug- 
gestions for future studies. 

Conclusion: It can be either a separate section or integrated with the discussion 
(always at the end of the discussion). It is usually made up of a few sentences 
summarising the key findings relating to the aim of the research/ project. 
Appendix: Provides extra supporting information in relation to the report, 
such as diagrams, charts, sample questionnaire, calculations, and raw data 
(e.g. raw SPSS data). 

References: A list of all the sources referred to in the report. It is presented 
in alphabetical order with full bibliographical details of the sources used in 
the report. 


Advantages of Report Writing 


Bw hd 


Learn techniques in data collection, analysis and reporting. 

Develop judgments about experiment procedures, results and limitations. 
Enhance writing skills in presenting practical work. 

Assess different levels of cognitive knowledge. 


Disadvantages of Report Writing 


1 


Time-consuming assessment approach for both students and teachers. 


How to Design a Good Report Writing Assessment? 


1 


If you require students to follow a specific report format, it must be clearly 
specified. Different disciplines may have their own style of format. 

Remind students to be aware of the use of technical terms and symbols. People 
from other disciplines may not be familiar with those terms. Therefore, stu- 
dents are required to provide definitions of the technical terms and symbols. 
If English grammar and language are part of the marking criteria, this must 
be clearly specified. 

Provide some guidelines on reporting numbers, units of measurement and 
scientific diagrams for students. 

Provide a report checklist for students. There are different sections in a re- 
port format, and each section has its specific content in it. A checklist would 
help students include all the essential content in each section. 

Make sure students understand the assessment criteria. Students have to be 
aware what aspects are going to be assessed, such as the writing style, anal- 
ysis, diagrams and referencing. 

Timely feedback is important for improvements. Feedback mechanisms such 
as mid-term report or pre-writing meetings will help to guide students before 
the final writing report is due (Table 4.10 shows a sample rubric to assess stu- 
dents’ written reports on their problem-based learning in a group project.). 
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4.3.2 Experiential Learning Activities 


In this section, common experiential learning activities conducted in universities 
around the world (i.e.) Capstone project, community service, field trips, intern- 
ship, student exchange programmes, entrepreneurship education programmes, 
student organisations and residential education are presented together with the 
common assessment approaches employed as short cases. 


Capstone Project 


In Reinventing Undergraduate Education (Boyer Commission, 1998), capstone 
experience is defined as an authentic project or activity organized based on all the 
knowledge and competencies that a student has gained and developed during the 
course of his/her studies. The purpose of a capstone project is to apply what has 
been learned in the programme, culminating the experience, the knowledge and 
the competencies gained into one evidencable assessment. At the University of 
Hong Kong (Chan et al, 2017), capstone experience may be in the form of a re- 
search project, research paper, thesis, portfolio or internship. Capstone experience 
broadens student’s perspective and allows students to take responsibility and own- 
ership for their learning. Students can explore knowledge beyond their studies and 
learn to work individually or collaboratively with or without faculty supervision. 
These experiences are great assessment grounds in providing the faculty or de- 
partment information on the strengths and weaknesses of their curricula because 
students’ performance is being observed and assessed holistically, and it provides 
a true method of direct evidence of student learning. Capstone experience can be 
a data-rich vehicle for programme assessment, to evaluate how well students have 
achieved the programme outcomes. According to Hartmann (1992), students’ 
research paper is “an unwieldy basis for external assessment, they provide the most 
direct and most unfiltered picture of students’ capabilities” (p. 128). 


Frameworks for a Capstone Experience (from Rowles et al., 2004) 


Capstone experiences can be organized with respect to these four frameworks 
that target the capstone programme’s needs. Although, one framework is gen- 
erally adopted, other frameworks may also be incorporated or acknowledged 
where appropriate. 


l Mountaintop: Interdisciplinary or multidisciplinary capstone experience, 
in which students from two or more disciplines work collaboratively on the 
same project. Such capstone projects mirror the real-world setting, whereby 
students are required to work with people from diverse background. 

2 Magnet: Discipline-specific capstone experience which requires students to 
draw upon concepts learnt from various courses in the discipline. Such capstone 
projects act as a “magnet” that pulls together knowledge in a summative man- 
ner, requiring students to produce a final product to demonstrate their learning. 
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3 


Mandate: Students enrol in capstone courses that are organized to meet 
the standards and requirements set out by external industry or professional 
bodies (e.g. ASCE, AACSB, ABET). 

Mirror: Capstone courses which requires students to reflect on their ex- 
periences and metacognitive skills in relation to programme objectives and 
goals. Through reflective writing, students may describe what they have 
learnt and how their assignments and experiences have helped them achieve 
each of the expected learning outcomes. 


List of competencies that can be developed 


Aside from discipline-specific knowledge and skills, well-designed capstone pro- 
jects can promote development of the following competencies and attitudes: 


` QD 0714 wd ra 


Interpersonal 
Communication 
Teamwork 

Problem solving 
Self-understanding 
Critical thinking 
Information literacy 


This is by no means an exhaustive list of competencies. 


Common Assessment Methods for Capstone Project 


Report 


A report is a systematic and well-organized document that defines and analyses 
certain topic or problem related to the project. Depending on the type of pro- 
ject, students may be asked to prepare a report (e.g. a laboratory report for a sci- 
ence project, a technical report for a design project). For some courses, students 
may be required to submit an interim progress report before the final report as a 
form of formative assessment. 


Case Example: Marketing Research course at the 
University of Hong Kong (The University of Hong 
Kong HKU Business School, 2021) 


In an undergraduate Marketing Research course (MKTG3502) offered 
by the Faculty of Business & Economics at the University of Hong Kong, 
students form groups to conduct a marketing research project that weights 
40% of the final. In the form of a marketing research report, students 
are required to demonstrate their ability to identify a marketing related 
topic/question, design research, perform research with a software taught 
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in class, and present findings with a marketing plan. Students are assessed 
with a rubric based on the quality of the executive summary, appropriate- 
ness of data analysis procedures, quality of the recommendations, and the 
effectiveness of both presentation and writing. The rubrics is provided to 
the students for clear guidelines 


Oral Presentation 


During an oral presentation, students demonstrate not only their understanding 
and knowledge of their project outcomes, but also their ability to effectively 
communicate their ideas to a group of audience. It is also common for students’ 
presentation to be followed by a discussion session where their supervisor and/ 
or peers ask questions on the presented materials. 


Case Example: Introduction to Embedded Systems 
course at the Hong Kong University of Science and 
Technology (Hong Kong University of Science and 
Technology, 2021) 


The “Introduction to Embedded Systems” course (ELEC3300) offered 
by the Department of Electronic & Computer Engineering at the Hong 
Kong University of Science and Technology is an elective, which gives stu- 
dents the flexibility to work in pairs on a project of their choice. Based on 
a rubric, students’ oral presentation on their final project demonstration is 
assessed on clarity, creativity, originality and level of understanding. 


Portfolio 


A portfolio is a documentation ofa student’s learning achievements. A portfolio for 
project work may include students’ major pieces of work, feedback from their su- 
pervisor, and their own reflective analysis. A complete and well-organized portfolio 
can show a student’s development and learning progress throughout the project. 


Case Example: E-portfolio for Final Year Project 
at the Hong Kong Polytechnic University (Chan & 
Yan, 2013) 


The Hong Kong Polytechnic University adopted e-portfolio as a formative 
assessment method for the final year project of engineering students. A ru- 
bric is developed to assess students on their choice of artefacts, reflection/ 
critique, use of multimedia and a variety of resources, overall layout and 
quality of writing. 
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Peer Assessment 


Peer assessment allows students to assess each other’s performance in the project. 
It can be extremely valuable in helping students to learn from each other, and to 
become responsible for their own learning. 


Case Example: Final Year Plant Design Project 
at Imperial College London (Imperial College 
London, 2016) 


The Final Year Plant Design Project offered by the Department of Chem- 
ical Engineering at Imperial College London requires students to work in 
teams of approximately ten students to design a process plant. Students 
are expected to submit periodic peer assessment online, so that they can 
reflect on their individual contributions and teamwork skills, gain a better 
understanding of how their team works, and actively consider whether 
anything can be done to improve team effectiveness. 


Project Audits and Reviews 


Project audits refer to the evaluation of team progress. As a standard practice in var- 
ious industries, it is designed to offer formative feedback for teams to reach project 
goals under qualified and systematic project governance in order to enhance project 
works. Project audits include three stages (Audit 1: Concept of Operations; Audit 
2: Mid-term Project Audit and Audit 3: Final Project Audit). It begins by setting up 
the project agenda and scope (Audit 1). In the second stage (Audit 2), the process 
is guided and evaluated according to the project scope. Finally, the audit process is 
finalised, and is prepared to move to another stage of the project (Audit 3). 


Professional Reflection 


Professional reflection refers to the activity adding to one’s progressing profes- 
sional engineering development. It is designed to allow people to reflect on their 
learning experience and regard this as a launching point for the new stage of 
their career. To present professional reflection, we can adopt one of the follow- 
ing six options, including (1) Application Package; (2) Professional Reflection/ 
Portfolio; (3) Technical Leadership Program; (4) Engineering Logbook; (5) 
Continuing Professional Development and (6) Professional Accreditation. 


Case Example: Capstone Systems Engineering 
Project at Australian National University 
(Australian National University, 2021) 


The Capstone Project offered by ENGN4221 Systems Engineering Project 
at Australian National University requires students to complete relevant 
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assessment activities that help practice evaluating the process of projects. 
The assessment activities include three project audits and reviews (con- 
ducted in Week 3, 6 and 10) that weight 75% of the Capstone Project and 
a professional reflection that weights 25% of the project. Activities done in 
each audit week are the same. On Monday of each audit week, each project 
team is required to submit a “Landing Page” and the project review on 
Wattle. The deadline of submission is on Friday of the same week. During 
the tutorials in the audit week, students present their project team pro- 
gress in open-format. On the Wednesday after each audit week (Week 4, 
7 and 11), the feedback on the project review will be available via Wattle. 
During the tutorials in these weeks, a progress indicator is given for each 
audit progress and the project teams plan their actions according to the 
feedback received. For the final audit grade, students are assessed based on 
the project audit grade, team member contribution weighting and project 
review weighting. 

Apart from project audits and reviews, students are also required to 
prepare for the professional reflection. In the tutorials of Week 7, apart 
from planning further actions based on feedback received from the mid- 
term project audit, students need to finalise the option of the professional 
reflection from the six introduced. After students showcase their projects 
through presenting their showcase posters and submit the final team and 
project reviews, they need to submit the professional reflection on Monday 
in Week 13. For the professional reflection, students are assessed according 
to the following three criteria: 


e Professional approach and attitude 
e Evidence of learning 
e Maturity of reflection 


Community Service 


Service-learning holds up a MIRROR for us to see ourselves, a MICRO- 
SCOPE for us to examine our society, and BINOCULARS for us to see 
what lies ahead. - Mark Cooper 


Through engaging students in improving and solving social problems, commu- 
nity service promotes both intellectual and civic engagements by relating aca- 
demic learning to real-world problems and needs. Community service projects 
can vary in terms of destination (i.e. overseas or local), nature of work and credit 
allocation. They can be initiated by external organisations, higher education 
institutions or the students themselves. 

Some people may argue that assessing service learning may diminish the 
meaning of participation (Chan, 2012a; Chan & Yeung, 2021); however, assess- 
ing students is important in order for them to be aware of and reflect on their 
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learning experience, and the trick is to assess the meaningful parts of the expe- 
rience for enhancing student learning using appropriate assessment approaches, 
and not just assess for the purpose of quality assurance. Assessment also provides 
“evidence for faculties to support their belief that service-learning has a rightful 
place in the program” (Strouse, 2003, p. 78). This is important particularly as 
there have been many well-cited research literatures indicating that experiential 
community service-learning projects allow different holistic competencies to de- 
velop (Astin & Sax, 1998; Chan, 2012b; Kuh, 1995). 


List of competencies that can be developed 


Through engagement in community service, students will be able to develop the 
following holistic competencies: 


Interpersonal 

Communication 

Teamwork 

Problem solving 

Self-understanding and resilience 

Leadership 

Empathy and respect (Weah et al., 2000) 

Adaptability 

Cultural Sensitivity and Global Citizenship (Weah et al., 2000) 
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This is by no means an exhaustive list of competencies. 


Common Assessment Methods for Community Service 


Reflective Journal 


Reflective journal helps students make sense of their community service expe- 
rience by providing them the opportunity to critically evaluate and recap their 
understanding from what they have seen and done. Reflective journal tends to 
be more effective when students are given prompts to guide them towards eval- 
uation and reflection instead of giving mere description. It can be a daily/weekly 
record of student’s reflection. 


Case Example: Community Service-Learning 
Projects at the University of South Australia 
(University of South Australia, 2016) 


The Community Service-Learning Project (EDUC4186) is an elective 
for all students at the University of South Australia, in which students 
work in multi-disciplinary teams to help community organisations solve 
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real problems based on their needs. Students are required to submit a 
1,000-word reflective journal as part of their assessment on their use of 
knowledge and application of skills taught in their degree, as well as the 
relevance of the community service experience to their future career. 


Direct Observation 


Direct observation is conducted by teachers and/or supervisors on students’ per- 
formance during the community service project; it is based on observers’ judg- 
ments of students’ ability in relation to the intended learning outcomes. 


Case Example: Sichuan Reconstruction Community 
Service Project organized by the Faculty of 
Engineering at the University of Hong Kong 
(Chan, 2015) 


In addition to other assessments, such as presentation, daily reflective 
journal and summative report, the daily performance of students partici- 
pating in the Sichuan Reconstruction Community Service Project organ- 
ized by the Faculty of Engineering at the University of Hong Kong was 
assessed by both professional assessors in Sichuan and team supervisors. 
Team supervisors recorded their observation of students’ daily perfor- 
mance (e.g. students’ initiative to learn and contributions to the pro- 
ject), while professional assessors in Sichuan recorded their observation 
of students’ team building activities (e.g. assessment of how well students 
participated in and cooperated with group members during the service 
project) (Chan, 2012b). There were a number of critiques of using direct 
observation as an assessment, including the logistics of collecting the 
observation records from team supervisors and professional assessors, and 
the personal relationships between the assessors and the students who 
may became closed as they were both engaged in the community service 
activities. In Chapter 9, we will demonstrate using technology to facili- 
tate many logistics challenges and in Chapter 7, ethics in assessment will 
be discussed. 


Oral Presentation 


Community service-learning experience is usually authentic and meaningful. 
Presentation allows teachers to not only evaluate students’ applied knowledge 
and competencies via the community service-learning experience but also their 
enthusiasm towards the experience. This form of sharing with teachers and peers 
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may help fellow students relive and reflect on their own experiences. Students 
may be assessed in terms of content relation, knowledge grasp, presentation 
style, enthusiasm, and audience engagement. It is important that teachers pro- 
vide clear criteria on what are to be assessed for oral presentation so students 
spend the time focusing on the expected outcomes. 


Case Example: Engineering Projects in Community 
Service (EPICS) Program (Purdue University, 2020) 


Real Design, real people and real impact are the slogans of Engineer- 
ing Projects in Community Service (EPICS) Program at Purdue Univer- 
sity. EPICS was founded in 1995, it features a unique programme where 
undergraduates work in teams to design and build engineering-related 
systems to solve real problems for the local community or education or- 
ganisations. EPICS fulfils the mutual needs of students to have authentic 
learning experience and of the local community to access the expertise 
that would normally be costly. 

EPICS targets a range of learning outcomes as reflected in its rich as- 
sessment methods for students’ projects and students themselves. Among 
all these outcomes, communication skills are attached with great impor- 
tance and oral presentation has been adopted as a major means to evalu- 
ate and promote students’ communication. For example, in the “Design 
Review Presentation”, students are expected to present their project 
designs to external design reviewers (e.g., EPICS advisors; industry ex- 
perts), explaining clearly and concisely how the project was designed, 
how it could be implemented in the future, and the potential risks in- 
volved. Oral presentation mimics the real design reviews every engineer 
practice regularly in the industry, and thus, apart from enhancing stu- 
dents’ communication skills, it provides very valuable opportunities for 
students to demonstrate their projects, to understand real engineering 
practices, and to establish connections with stakeholders. Similarly, oral 
presentation has also been used as a method to assess students’ commu- 
nication with their project partners. Alongside some casual meetings and 
feedback, oral presentation effectively reflects students’ oral competency 
to negotiate the needs from their partners in a formal way. A rubric on 
communication skills, alongside the other learning outcomes (e.g., team- 
work, critical thinking), has also been developed to guide students in 
their oral presentations. 

For these reasons, oral presentation not only serves as a powerful tool 
to assess students’ holistic competencies — including but not limited to 
communication skills, application of knowledge and networking skills, but 
also to drive student learning under a more communicative context which 
is highly valued in today? world. 
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Essay 


An essay can be used to help students increase their awareness of social issues, 
by integrating academic enquiry with practical experience gained through the 
community service experience. For example, students may be asked to identify a 
problem or an issue related to their community service experience, to critically 
analyse it by applying relevant theories that they have learnt from their discipli- 
nary studies, and to reflect on how their contributions impact on society as a 
whole through their experience. 


Case Example: Social Innovation and Global 
Citizenship Program at the University of Hong 
Kong (The University of Hong Kong Faculty of 
Social Sciences, 2021) 


Students who participate in the Social Innovation and Global Citizenship 
Internship at the University of Hong Kong are required to write an inte- 
grated essay that covers: (1) background information on the community 
organisation, (2) identification of service gap and making recommenda- 
tions/suggestions to the organisation, (3) description and evaluation of 
the service tasks in the service project, and (4) directions/recommenda- 
tions for future service projects for sustainable development. This type 
of essay requires students to plan and research on the area to which the 
service will be provided. The students would also require to think beyond 
what is needed as they will need to consider the clients, the environments 
and the community in which the service will be served. 


Blog 


A blog is a form of online journal where one can publish a series of entries 
with photos and videos (Farmer et al., 2008) on their community service ex- 
perience, anytime and anywhere. Students often encounter different scenarios 
during their community service experience. They can use the blog to capture 
these moments, and provide a personal account of it. It helps them to reflect and 
learn from it. 


Case Example: Transformative Business Immersion 
in Developing Economics course at the University 
of Hong Kong (The University of Hong Kong 
Business School, 2021) 


The Transformative Business Immersion in Developing Economics course, 
offered by the Business School at the University of Hong Kong, brings 
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students on a three-week field trip to the rural Philippines where they 
work with small local businesses. Through interviews with the owners and 
employees of the local businesses, students need to identify potential areas 
of improvement and to offer suggestions to help the businesses accomplish 
their financial goals. During their time abroad, students write a blog post 
once a week to share their homestay experiences, their relationship and 
project progress with the small businesses, and a personal reflection. Upon 
the completion of the trip, students are also required to produce a video 
showcasing their experiences during their trip. 


Field Trips 


Educational fieldtrips are used to augment classroom learning by taking students 
outside of the classroom to participate in exploratory activities at a featured loca- 
tion. There are many potential destinations, such as institutions like museums, 
community centres, non-governmental organisations (NGOs) and workplaces, 
or places like historical, geological or biologically diverse sites. As a type of ex- 
periential learning, fieldtrips can be engaging and enjoyable while also exposing 
students to specific aspects of the real world and allowing them to gain unique 
first-hand experiences. Fieldtrips can vary in length, from a few hours to a day, 
sometimes it may require overnight stays, or even weeks if the destination is in 
a different country. 

According to DeWitt and Storksdieck (2008), learning outcomes of fieldtrips 
are influenced by a number of factors, including the structure of the trip, the 
quality of the preparation and follow-up activities, the novelty of the destination, 
the interests of students, the social contexts involved, and the experiences of stu- 
dents during the trip. Learning can be cognitive (e.g. facts and concepts) and 
affective (e.g. sparking interest or changing attitudes towards a certain topic). 


List of competencies that can be developed 


Aside from discipline-specific knowledge and skills, well-designed fieldtrips can 
promote development of the following competencies and attitudes. 


Self-understanding 

Teamwork 

Observational competencies 

Critical thinking and the ability to synthesize information 
Curiosity and awareness 
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This is by no means an exhaustive list of competencies. 


Designing Experiential Learning Assessment 133 


Common Assessment Methods for Field Trips 


Quiz 

Quizzes can objectively assess knowledge relevant to or gained from fieldtrips. 
A quiz often includes questions with definite answers (e.g. multiple-choice 
questions) or short answer questions (e.g. fill-in-the-blanks, short descriptive 
answers). The questions are often directly relevant to what students are ob- 
serving and experiencing and students often need to fill in the answers during 
the fieldtrip. Thus, it usually takes a short amount of time to complete and 
mark. 


Case Example: BIOL3628 Marine Ornithology at 
Dalhousie University (Dalhousie University, 2015) 


The Marine Ornithology course, offered by the Biology Department of 
Dalhousie University, includes a one-day fieldtrip to seabird breeding col- 
onies in the Maritimes. Students are given a field quiz during the trip on 
seabird identification. 


Worksheet 


Similar to quizzes, worksheets assess knowledge related to fieldtrips and often 
include short answer questions. Students may be given the whole day to com- 
plete the questions, often finding the answers as they proceed with the different 
areas or segments of the trip. Students may sometimes have the option to discuss 
and compare their answers with their peers. 


Case Example: Understanding Climate Change at the 
University of Hong Kong (Faculty of Science, 2016) 


In the Understanding Climate Change common core course offered by 
the Faculty of Science at the University of Hong Kong, students partic- 
ipate in a half-day fieldtrip to Lai Chi Chong, a local site of geological 
significance. The aim of the fieldtrip is to have students see how geological 
records are used to discover environments and climates from the past. 
During the trip, students are given a worksheet to complete by the end of 
the day, including questions related to specific parts of the site and their 
significance. 

(Porter et al., 2012) 
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Field Notebook 


Students may be required to take observation notes during a fieldtrip, which can 
then be collected for marking and assessment. The notes may be qualitative and/ 
or quantitative. A student may jot down notes, take photos and record videos of 
their experiences and thoughts during the trip. Field notebook assesses student’s 
process, at the same time checking their understanding. The field notebook is 
often used together in a bigger report. 


Case Example: Montreal - Its Urban, Cultural & 
Social Practices at the Western University of 
Ontario (The University of Western Ontario, 2015) 


In the course ‘Field Methods and Practices: Montreal — Its Urban, Cul- 
tural & Social Practices’, offered by the University of Western Ontario, 
students travel to Montreal to explore the city’s geography and develop 
their research and observational skills. During the six-day trip, students 
are required to record their observations and experiences in a notebook, 
additional assignments are also included, which they hand in at the end of 
the trip for assessment. 


Written Report 


The contents of a report may include descriptions, interpretations, and analyses of 
a topic or investigation from the fieldtrip. Reports usually include an introduction, 
procedures, results, discussion and conclusion. While they can also vary in length, 
reports assess different levels of cognitive knowledge and critical thinking, such as 
requiring students to evaluate what they have observed during the trip and prompt- 
ing them to apply theories that they have learned in class to their experiences. The 
written report works in complement with the field notebook assessment, where the 
field notebook allows assessing of process while the written report assesses results. 


Case Example: Building Maintenance at the 
Universiti Tun Hussein Onn Malaysia (UTHM) 
(Interviewed with the teacher) 


In the Building Maintenance course at UTHM, students participate in 
a one-day field trip and visit a commercial or government building with 
a variety of mechanical, electrical and intelligence systems. Working in 
groups, they are assigned specific sections of the building on which they 
would write their technical report. The report includes a literature review, 
critical evaluations of the problem, and discussions on their suggestions 
for improvement projects. 
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Oral Presentation 


Oral presentation is a way for students to show and explain the contents of their 
fieldtrip and its specific topics to a group of audience. These presentations may 
include visuals and PowerPoint slides as aids, and there is often a questions- 
and-answers session after the presentation. 


Case Example: EBIO327 Biological Diversity Lab at 
Rice University (Rice University, 2015) 


The lecture/ laboratory course, Biological Diversity Lab, is offered at Rice 
University and involves a weekend-long fieldtrip to the Big Thicket Na- 
tional Preserve. Students work in groups and conduct their self-designed 
surveys for sampling taxonomic groups in the preserve. They are then 
required to submit a final report and give an oral presentation to the class, 
describing the results of their survey. 


Poster Presentation 


A poster presentation requires students to produce concise written content and 
clear visuals (e.g. photographs taken during the fieldtrip) on a poster for dis- 
play. The assessment should also consider the layout of the poster, present the 
information in a logical manner, all while making it creative and eye-catching. 
Students may sometimes give a supplementary oral explanation of their poster. 


Case Example: GLY330 Geomorphology at 
Northern Kentucky University (Northern Kentucky 
University, n.d.) 


The Geomorphology course at the Northern Kentucky University in- 
cludes several weekend field trips when students travel to Carter Caves, the 
Red River Gorge and up to three additional locations, to collect and ana- 
lyse data on the landforms and their processes of formation. Students are 
required to take notes and photographs at every trip and complete a group 
research project that includes the analysis of collected data, results and 
conclusions. Each project is shared to the class in a poster presentation. 


Reflective Journals 


When writing reflective journals, students record their thoughts and ideas on 
their experience and what they have gained or learned from the trip. Writing 
reflections allows them to express their feelings regarding their experiences while 
exercising their self-awareness and critical thinking. 


136 Designing Experiential Learning Assessment 


Case Example: FScN2001 Healthy Foods, 

Healthy Lives — A Food System Approach to 
Cooking at the University of Minnesota (University 
of Minnesota, 2016) 


In the ‘Healthy Foods, Healthy Lives — A Food System Approach to 
Cooking’, offered at the University of Minnesota, students participate in 
two fieldtrips to learn about food production and food culture. They are 
required to write a personal reflection for each of the trips, and submit it 
via a blogsite. 


Internship 


Internship provides students with the opportunity to gain valuable work experi- 
ence in the field they have chosen and to enhance their prospects for employment 
before graduation. Although many universities organize internship for their stu- 
dents as part of their undergraduate programme, students may also participate 
in an internship on a voluntary basis during their semester vacation to gain work 
experience. Depending on the decision of the individual faculty or department 
in the university, an internship could be credit bearing or non-credit bearing. 

The nature or type of work allocated and the availability of supervision at the 
workplace are some of the factors influencing students’ learning through intern- 
ship experience (Luk & Chan, 2020; Martin, 1997). As the experience of each 
student intern may vary significantly from each other, the learning outcomes 
of an internship may be quite different (Crebert et al., 2004; Hu et al., 2009). 
In the study conducted by the author's postdoctoral researcher, Lillian (Luk & 
Chan, 2021) on internship, she addressed the research question — what are stu- 
dents’ perceived learning outcomes from their internship experience? 

In her study, she revealed four categories of learning outcomes (as below), 
which form a provisional framework illustrating diversity in students! perceived 
learning outcomes of internship. 


Knowledge 

Academic-related generic competencies 
Non-technical generic competencies 
Technical competencies 


ance, 


She found that some learning outcomes (i.e. general learning outcomes) such 
as disciplinary knowledge and problem-solving skills, are more likely to be de- 
veloped than others (i.e. specific learning outcomes) such as research skills and 
professional judgement. The specific learning outcomes are unique to each indi- 
vidual student, and these outcomes may not be consistent for each student even 
if the students were in an identical role and company. While the idea of construc- 
tive alignment (Biggs, 1996) in course design suggests that learning outcomes 
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should be specified at the beginning of the course design, specific learning out- 
comes may be unknown in internship and (maybe) other experiential learning 
courses due to the nature of the activities. 


List of competencies that can be developed 


Through engaging in an internship, students will be able to develop the follow- 
ing competencies in a real-world workplace environment: 


Interpersonal 
Communication 
Writing 

Problem solving 
Self-management 
Adaptability 

Time management 
Work attitude and ethics 
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This is by no means an exhaustive list of competencies. 


Common Assessment Methods for Internship 


Survey 


A survey consists of rating scales and open-ended questions, which enables the eval- 
uation of students’ performance in an internship in both the students’ and the em- 
ployer's perspectives. It can be considered as a fairly objective assessment method. 


Case Example: Social Innovation and Global 
Citizenship Programme at the University of Hong 
Kong (The University of Hong Kong Faculty of 
Social Sciences, 2021) 


The Social Innovation and Global Citizenship Programme, offered by the 
Faculty of Social Sciences at the University of Hong Kong incorporates 
a performance appraisal form to assess the student intern's performance. 
The performance appraisal form includes rating scales for the employer to 
rate the student intern's level of adaptability, communication skills, work 
attitude, independence, motivation and self-management skills. 


Reflective Journal 


Reflective journal is a written series of students! ideas, personal thoughts, ex- 
J P ghts, 
periences, reflections, and insights gained during and after their internship. It 
provides a record of students' internship activities. 
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Case Example: One-year Undergraduate Internship 
at Aston Business School (Aston University, n.d.) 


Business undergraduates in Aston Business School who participate in a 
one-year internship are required to keep a reflective journal on their pro- 
gress, using PebblePad (http://www.pebblepad.co.uk). They are expected 
to submit monthly entries, which cover (1) their roles and responsibilities, 
(2) learning outcomes, (3) reflection on any problems faced and actions 
taken to resolve the problem, and (4) an outline of objectives for the next 
month. 


Report 


A report is a systematic and well-organized document that provides a descriptive 
account of an internship experience. It usually contains different sections, such 
as company background, job achievements, and learning outcomes. 


Case Example: Industrial Attachment Programme 
at the National University of Singapore (National 
University of Singapore, n.d.) 


The Industrial Attachment Programme offered by the Faculty of En- 
gineering at the National University of Singapore is a 12-credit course 
designed to expose students to day-to-day work environment of a pro- 
fessional engineer, so as to instil the students the right kind of work 
attitudes and professionalism through interactions and networking with 
practitioners. Student interns are required to submit an internship report 
on the internship progress every six weeks during the period of attach- 
ment, resulting in a total of five reports. In the reports, students are ex- 
pected to provide a detailed description on the type of work performed, 
relationship with their supervisors and colleagues, and their perceived 
learning outcomes. 


Portfolio 


A portfolio is a documentation of a student’s past achievements. In the context 
of an internship, the portfolio often includes students’ major pieces of work, 
feedbacks from their supervisor/industrial tutor, and their own reflective analy- 
sis. A complete and well-organized portfolio can show a student’s development 
and learning progress in the internship. 
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Case Example: The MyPortfolio system used by 
the University of Auckland’s Teacher Education 
Program (San Jose, 2017) 


The MyPortfolio system, hosted by Mahara, was adopted by the Uni- 
versity of Auckland’s Teacher Education Programme to support student 
teachers’ learning during their practicum. Student teachers specialising in 
early childhood education are encouraged to document different aspects 
of their practicum experience (e.g. a journal entry on a parent/mentor 
meeting session, photos of the classroom learning environment, a video re- 
cording of children learning in class) in the e-portfolio. They are expected 
to critically review their practice based on the documentations and try to 
come up with ideas to enrich the overall teaching and learning experience. 


Oral Presentation 


An oral presentation is rarely a standalone assessment and it is often used to 
accompany other assessments such as a report, a portfolio, etc. During an oral 
presentation, students demonstrate not only their understanding and knowl- 
edge of their internship/placement outcomes, but also their ability to effectively 
communicate their ideas to a group of audience. It is also common that the 
student's presentation is followed by a discussion section, where the academic 
and/or work supervisor may ask questions and the student needs to give sensible 
response and defend any presented materials. 


Case Example: The Real-World Internships and Projects 
at Queensland University of Technology (QUT, n.d) 


The Real-World Internships and Projects (AMB310) course is an elective 
for business students in their final year of study at Queensland University 
of Technology. Students enrolled in the course are required to complete a 
minimum of 120 hours of work placement during a 13-week semester. The 
poster presentation, as an assessment task, was introduced in year 2009, 
to replace a written report and individual class presentation. Based on the 
goals and activities set in their internship plan, students are expected to 
give an oral presentation to provide a review of the internship, a reflection 
on their learning experiences and development of capabilities. The posters 
are assessed based on the following criteria: 


° Poster presentation (visual appeal, clear layout, professional presenta- 
tion, meeting size guidelines) 

e Poster content (identifying student and organisation, using reflective 
writing frameworks to structure content) 

e Audience engagement (presentation, discussion, behaviours) 
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Logbook 


Logbook contains a record or proof of a student’s learning accomplishments, 
learning experience, and learning outcomes. Students can include their logbooks 
in their CV for future job-seeking purposes. 


Case Example: Industrial Attachment Program at 
the National University of Singapore 


The Faculty of Engineering at the National University of Singapore re- 
quires engineering students to complete a student log sheet during their 
Industrial Attachment Program. This log sheet is updated weekly and has 
to be attached to the student’s final report at the end of the internship. 
Basically, the log sheet records the number of weeks, work(s) performed 
by the student, and the supervisor’s/ industrial’s signature and comments 
for verification. 


Learning Contract 


Learning contract is drawn between students and their supervisor/industrial 
tutor to develop learning plans. Learning objectives, strategies, and expected 
learning outcomes are negotiated, with the student’s competencies and the su- 
pervisor’s/ industrial tutor’s expectations taken into account. 


Case Example: Credit-bearing Internships at 
the University of Kentucky (the University of 
Kentucky n.d.) 


The University of Kentucky requires students to submit a learning con- 
tract in order to receive credits for their participation in the internship 
programmes offered by different academic departments. To complete the 
learning contract, students need to identify an academic advisor in their 
academic department, with whom they discuss learning objectives and 
assignments. Students are also required to state specific time and dates on 
which they plan to have periodical meetings with the academic advisor to 
provide updates on the internship progress. 


Exchange Programme 


Student exchange programme provides the opportunity for students to spend 
at least a semester in an overseas university, allowing them to enrol in academic 
courses offered by the host university and to gain an understanding of the 
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language and culture of the host country (Sowa, 2002). The way students are 
assessed during an exchange programme is often dependent on the course(s) 
selected. Most of the time, students are involved in in-course assessments, rather 
than an assessment of their learning from the exchange experience as a whole. 
In Luo & Chan (2022)’s systematic review paper, many qualitative methods to 
assess intercultural competency are revealed. 


List of competencies that can be developed 


Through engagement in an exchange programme, students will be able to de- 
velop the following competencies: 


1 Interpersonal Communication 
2 Self-management 

3 Adaptability 

4 Intercultural competencies 

5 Language 


This is by no means an exhaustive list of competencies. 


Common Assessment Methods for Exchange Programme 


E-portfolio 


An e-portfolio is an electronic documentation of students’ work to showcase 
their learning. In the context of an overseas exchange programme, an e-portfolio 
may include a reflection on their participation in different cultural activities 
in the host country, an identification of real-life problems/ issues in the host 
country and suggestions on how to deal with them, etc. A complete and well- 
organized portfolio can show a student’s development and learning progress 
during the exchange programme. 


Case Example: Exchange Experience Assessment 
offered by the Department of Information Systems 
at City University of Hong Kong (City University of 
Hong Kong, 2016) 


The Exchange Experience Assessment (184935) offered by the Depart- 
ment of Information Systems at City University of Hong Kong gives stu- 
dents the opportunity to spend two semesters in an overseas university. 
Students are required to prepare an e-portfolio which includes a two- 
minute video demonstrating their understanding of cultural differences 
and their overall evaluation of the exchange experience. Students are also 
encouraged to visit their peer's e-portfolio page to give comments. 
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Entrepreneurship Education 


Universities are increasingly interested in entrepreneurship education pro- 
grammes due to the demand and interest in new job opportunities and devel- 
opment of a knowledge-based economy. Entrepreneurship education, in the 
form of programmes, events or services regarding entrepreneurship, encour- 
ages students to pursue entrepreneurship-related knowledge and abilities, and 
also to create start-ups. However, the learning outcomes of entrepreneurship 
education programmes are unclear, causing challenges in pedagogical design. 
Ms. Hannah Wong, my postgraduate student, has conducted a systematic re- 
view on entrepreneurship education and developed a framework for learning 
outcomes of entrepreneurship education within higher education (Wong & 
Chan, 2021). While students are believed to gain a range of achievements such 
as knowledge, mindsets and careers in self-employment, development of com- 
petencies is a vital component within entrepreneurship education programmes. 


List of competencies that can be developed 


Through enrolling in an entrepreneurship education programme, students will 
be able to develop the following competencies: 


Creativity 

Interpersonal competencies 
Problem solving 

Financial management competencies 
Teamwork 

Opportunity identification skills 
Negotiation capacity 

Marketing skills 
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This is by no means an exhaustive list of competencies. 


Common Assessment Methods for Entrepreneurship 
Education 


Authentic Assessment and Creative Assessment 


Authentic assessment and creative assessment are two assessment methods 
adopted in the setting of entrepreneurship education. Authentic assessment in- 
cludes assessments on authentic practices, which in entrepreneurship education 
is often related to practical and experiential assessments in an actual workplace. 
Creative assessment includes assessments that are creative and innovative, as en- 
trepreneurship education seeks to align student learning with its innovative, flex- 
ible and creative nature. 
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Case Example: Entrepreneurship education within 
higher education settings, from Wong and Chan 
(under review) 


Universities often adopt authentic assessment in entrepreneurship educa- 
tion programmes, in which students are demonstrating what they have 
learnt in a mock or authentic workplace environment. This may include a 
range of assessments, for example assessing students’ business ideas, busi- 
ness plans, proposals, reports and presentations, where assessments may 
be conducted in front of peers, teaching staff or entrepreneurial profes- 
sionals. These assessments may be interrelated, where reports or presenta- 
tions could be founded upon business ideas and plans, and it is believed 
that students are gaining authentic understandings and skills for future 
careers. 

Entrepreneurship education programmes are also looking to creative 
assessments, in which assessments could be gamified or incorporating new 
approaches such as visual logbooks or performances. Often designed in a 
mix of formative and summative form, creative assessment is believed to 
allow students to gain and track development of skills and competencies 
through the process of assessment, particularly developing innovation and 
creativity skills, while enjoying and engaging in the assessment methods. 


Student Organisations 


Student Organisations include every student groups, clubs, societies, unions and 
associations that are led by students. In each tertiary institution, students can 
link up with like-minded people to initiate an organisation and pursue their 
interest. The university usually provides a certain amount of financial support to 
each legitimated student organisation for its development. There are a variety of 
student organisations, with some examples below: 


l Student administrative bodies: Student Union, Union Council. 
Religious-based organisations: Christian Association, Buddhist Studies 
Society. 

3  Academic-based organisations: Engineering Society, Law Association. 

Interest-driven organisations (sport/culture): Music Club, Dance Club. 

5 Community-based organisations: Social Service Group and the Greenwoods. 


EN 


Student Organisations hold co-curricular or extra-curricular activities that en- 
rich and enhance students! campus life. Both the executive organising commit- 
tees of each student organisation and their members can gain various learning 
experiences from organising or joining the activities. 
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One of my doctorate students, Ms. Katherine KW Lee, has focused her re- 
search on exploring the key experiences that can enhance — or even diminish — 
the sense of leadership self-efficacy among university students. Self-efficacy, 
which is comparable to self-confidence (McCormick, 2001) and is defined as 
an individual’s own assessment of their ability to perform and succeed at a task. 

The students in Katherine’s study identified various experiential opportunities 
throughout both their secondary and tertiary education that they considered to 
be important to enhancing leadership development, including internships and 
work placements, community service activities, and extra-curricular research or 
competition projects. One particular experience that was notable among stu- 
dents in Hong Kong was joining an executive committee in university, such as 
for a student-led club or faculty society. Students talked about how this stood 
out to them as not only a unique experience, but also one that was intense and 
hands-on as they would be responsible for organising events and functioning 
with virtually no faculty or teacher supervision. The high demands of a typical 
executive committee also meant that members had to rely on one another to 
solve problems and quickly learn for themselves how to negotiate with corpo- 
rate sponsors and communicate with industry representatives. Overall, this type 
of experience provided students with a multitude of valuable opportunities, in- 
cluding an environment in which they can observe and learn from their fellow 
members, authentic and meaningful ways to learn and develop their leadership 
skills as well as other competencies, exposure to different situations and people 
outside of a typical university classroom, and the simple yet important opportu- 
nity to put their skills into practice, allowing them to build up experience and 
confidence as they reflect on both their strengths and areas for improvement. 


List of competencies that can be developed 


Within the student organisations, apart from learning a new field or hobby, student 
clubs, societies and organisations enable students to plan, coordinate, market for, get 
sponsorship and evaluate activities and projects, which develop many competencies. 

As executive committees of the student organisation, students will be able to 
develop the following competencies: 


Leadership 

Creativity 

Organising 

Financial management 
Social and communication 
Teamwork 

Critical thinking 
Cooperation 

Problem solving 
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As member/participants of the student organisation, students will be able to 
develop the following competencies: 
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1 Social and communication skills 
2 Personal growth 


This is by no means an exclusive list of competencies. 


Common Assessment Methods for Student Organisations 


Oral Presentation 


An oral presentation involves introducing and explaining certain details of the 
student organisation’s activities to a group of audience. Often, the content and 
presentation skills are assessed. 


Case Example: Campaigning for the Candidacy of 
Executive Committee Elections at the University of 
Hong Kong 


At the University of Hong Kong, every proposed cabinet who would 
like to become the executive committee of a student organisation needs 
to go through a campaign. During the campaign, the proposed cabinet 
needs to present two main details to the previous executive committees: 
(1) define and clarify the roles and duties of each proposed committee 
member; (2) introduce the proposed year plan, which includes the aims 
and objectives, content, potential target group and contingency plan for 
each proposed activity. 

Members of the past executive committees then take turn in asking 
questions to challenge the proposed cabinet, by pushing the proposed 
cabinet to demonstrate their leadership skills and team working skills. 
They also assess and comment on the proposed year plan regarding its 
feasibility. The aim of the whole campaign is to assess, prepare and equip 
the proposed cabinet to become the incoming executive. 


Residential Education 


Residential education programmes involve co-curricular and curricular activities 
for students to enhance their learning experiences and achieve their learning 
outcomes in their university living environment. Universities believe that stu- 
dents are able to develop various academic, social, values-related, and attitu- 
dinal outcomes via residential education programmes. These areas encompass 
the learning outcomes required for students to be highly successful and global 
citizens. There are formal programmes like high table dinners, community de- 
velopment and regular floor meetings, as well as informal activities like different 
cultural events, competitions and sports teams. The types of residential activities 
can be very diverse across halls/ colleges. 
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List of competencies that can be developed 


Participating in formal programme can help develop the following competencies: 


1 Respect 
2 Critical thinking 


Participating in informal programme (cultural and sports teams) can help de- 
velop the following competencies: 


l Social and Communication competencies 
2 Leadership and cooperation competencies 
3 Health and wellness literacy (safety, exercise) 


Additional competencies are developed as students leave home and live by 
themselves. 


Self-management competencies (self-discipline) 
Problem solving 

Financial Management competencies 
Adaptability 
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This is by no means an exhaustive list of competencies. 


Common Assessment Methods for Residential Education 


Peer- and Self-assessment 


Peer- and self-assessment are two formative assessment methods that are widely 
used in the setting of residential education. Peer-assessment allows students to 
provide and get feedback from others; while self-assessment allows students to 
reflect on their own performance and to see if they meet their standards or 
expectations. 


Case Example: Residential Education at the 
University of Hong Kong 


At the University of Hong Kong, each floor and every cultural teams and 
sport teams of all residential halls have a Mid-year evaluation (MYE) at 
the end of the first semester and an End-year evaluation (EYE) at the 
end of the second semester. Both MYE and EYE are face-to-face eval- 
uation including self- and peer-assessment. In a floor/team evaluation 
meeting, members take turns commenting on themselves and others. The 
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evaluation is conducted in two parts: individual evaluation (assessing one’s 
holistic competency development) and post evaluation (assessing one’s 
performance based on their roles — e.g. floor-chairperson, team captain, 
and team member). It is believed that by doing MYE and EYE, students 
can understand themselves more, learn from each other, and eventually 
improve themselves. 


Conclusions 


Assessing experiential learning activities may need to involve more than one assess- 
ment approaches. In the above examples, various innovative assessment pedagogies 
were employed. It is encouraged to look through the entirety of these examples via 
the references provided. In Chapter 8, some best practice cases have been selected to 
provide a detailed account of the assessment used (Tables 4.11—4.15). 


Questions to Ponder 


e Is it actually ethical to assess students’ experiential learning experi- 
ence? Or their reflection? 

e How can we measure students’ value added? 

e How can we assess students’ reflection? 

e How can we provide constructive and appropriate feedback? 

e What do we know about students’ prior experience? 

e How do we know what have students learnt from their prior experience? 

° Howdo we knowifstudents have improved in their holistic competencies? 


Personal Reflection 


One of the rationales for me to write this book is because I have come 
across a number of service-learning centres which employ reflective writ- 
ings to assess the service-learning activities. Students are usually required 
to submit a 500 words reflective essay as the assessment. In general, such 
assessment aligns well with their learning, particularly if students were giv- 
ing some guidelines and practices. However, issues arise in terms of who 
will assess the reflective essays, and will the students be given feedback? 

Often, the answers are “No” and “No”. The assessment is purely a bureau- 
cratic piece of evidence. And the worst part is that students also realise that, 
they condemn the assessment and query the cause, the assessment process 
and the service-learning experience suddenly become meaningless. This is sad. 
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Marking Rubrics — Worksheets 


Table 4.15 A marking rubric for students to assess students’ worksheets during their 


internship 
Marking Excellent 
Rubrics 
Grading 
Descriptors 


Proficient 


Average 


Poor 


Completeness - the worksheet 


Knowledge 
application 
and transfer 


Quality 


Writing 
mechanics 


is complete 
and all parts/ 
questions are 
addressed to 
the fullest 
extent. 


- shows advanced 


understandings 
of both 
academic and 
industrial 
knowledge; 
demonstrates 
effective 
application 
of academic 
knowledge 
to the work 
context. 

- the worksheet 
entries show 
in-depth 
reflection of 
the internship 
experience 
and are highly 
oriented to 
the student’s 
future career 
goals; 

- entries are very 
clear, logical 
and accurate. 


- language use 
is mature, free 
from errors. 


- the worksheet 
is complete 
however not 
every part/ 
question are 
addressed to 
the fullest 
extent. 


- shows 
comprehension 
of both 
academic and 
industrial 
knowledge; 
demonstrates 
application 
of academic 
knowledge 
to the work 
context. 


- the worksheet 
entries show 
reflection of 
the internship 
experience and 
are oriented to 
the student’s 
future career 
goals; 

- entries are 
mostly clear, 
logical and 
accurate. 


- language use 
is appropriate 
with minor 
errors. 


- the worksheet 
is mostly 
complete; only 
some parts/ 


questions are 
fully addressed. 


- shows limited 
understandings 
of both 
academic and 
industrial 
knowledge; 
little evidence 
of applying 
academic 
knowledge 
to the work 
context. 


- the worksheet 
entries show 
limited 
reflection of 
the internship 
experience and 
are marginally 
oriented to 
the student’s 
future career 
goals; 

- entries could 
be clearer, 
more logical 
and accurate. 

- improvement 
as there are 
apparent 
errors. 


- the worksheet 
is not 
complete; 
many parts/ 
questions 
are not fully 
addressed 
or even 
addressed. 

- shows no 
understanding 
of both 
academic and 
industrial 
knowledge; 
no attempt or 
poor attempt 
to apply 
academic 
knowledge 
to the work 
context. 

- the worksheet 
entries show 
no reflection 
of the 
internship 
experience 
and are not 
oriented to 
the student’s 
future career 
goals; 

- entries lack 
clarity, logic 
and accuracy. 


- language use 
is problematic 
with errors. 
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5 Reflection as Assessment in 
Experiential Learning 


Reflection is how you see yourself before, now and after; how you see yourself 
from different perspectives; how you see yourself after certain situations or expe- 
riences; how you see yourself, your actions and your behaviours after you observe 
others. 

— Chan, CKY 


Introduction 


This chapter is solely dedicated to reflection. As illustrated in Chapter 1, it is a 
must-have feature in experiential learning. While reflection is often used as an 
assessment in experiential learning, it is complex (Rodgers, 2002) and is very 
difficult for learners and teachers to employ and assess, thus, it merits a chapter 
by itself. 


5.1 Reflection is ... 


Reflection is an integral part of experiential learning. Dewey (1933) fittingly 
stated, “we do not learn from experience... we learn from reflecting on expe- 
rience.” Experience alone does not necessarily lead to learning; active and con- 
scious reflection on experiences, emotions, actions and responses is essential to 
make meaning out of them (Loughran, 2002; Tomlinson, 1999). But what ex- 
actly is reflection? If you ask a layman, the answer may well be — it is an image 
that you see in a mirror. This in its most simplistic way is exactly what reflection 
is; reflection is how you see yourself, but it does not end there. 

For the majority of circumstances in life, reflection comes instantaneously 
(i.e. doesn’t require us to reflect critically or intentionally). For example, a stu- 
dent, Sean, enrols in a carpentry apprentice programme, and accidentally he 
holds the sharp end of a saw and cuts himself. His brain will automatically make 
a connection that holding the sharp end of a saw can hurt him. However, there 
are also a large number of circumstances in life which requires deeper reflection 
as they are not always obvious. Let's use the same carpentry apprentice example 
and link it to academic learning. Apart from practical carpentry works, carpen- 
try courses often include carpentry mathematics such as geometry, building 
layouts, foundation work, roofing, stair construction, understanding carpentry 
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tools, siding and mouldings. Sean, the carpentry apprentice has to put into 
practice what he learns in his academic class, and that requires reflection. The 
apprentice will learn and experience, but he will not improve his skills without 
proper reflection. For example, Sean is asked to make a creative wooden piece 
for his project, a piece that he has to visualise on his own. As he has never seen 
such a piece created previously, he may not be able to decide which type of tools 
to use: would a handsaw be more suitable, or perhaps a circular saw? Sean needs 
to reflect on previous classes, bringing classroom knowledge together with the 
practical skills that he has gained. He needs to ask himself questions, how can 
I cut that piece with a handsaw, would that make smooth edges? Can I combine 
it with other tools? Previously, the teacher showed us how to cut corners using a 
circular saw, could I do the same for mine? Without reflection, he will be an 
absentee in developing new skills and will carry on doing things the way he has 
always been doing. Opportunities to develop creativity and critical thinking 
will be lost as he lacks the self-questioning attitudes needed to solve ill-defined 
issues and interpret meanings and feelings for himself. That said, most learners 
(people) think unconsciously, it is unusual that a learner does not perform some 
kind of reflection while experiencing an event, and the crucial differences lie 
in how systematic, rigorous, structured and constructive are one’s way of think- 
ing. And this is the type of reflection assessment that teachers should design 
within their experiential learning activities to train learners in their intentional 
reflection. 

According to Van Beveren et al. (2018), “the concept of reflection has been 
extensively researched and theorized” (p. 1) over the previous decades. As Kem- 
ber et al. (2008) pointed out, the concept of reflection is “widely and diversely 
used” (p. 369); written in different times and under different circumstances, 
many definitions and concepts of reflection have developed. This results in the 
differences in explanations and definitions of reflection, with various new terms 
branching from the idea of reflection. Below are some reflection concepts stud- 
ied by educational researchers. Dewey defined reflection as: 


An active persistent and careful consideration of any belief or supposed form 
of knowledge in the light of the grounds that support it and the further 
conclusion to which it tends. 

(Dewey, 1933) 


According to Boyd and Fales (1983), reflective learning occurs when an individ- 
ual’s internal exploration of an issue or experience helps them to create or clarify 
its Meaning. 


In 2002, Rodgers re-examined Dewey’s philosophy behind reflection and estab- 
lished four expanded concepts behind Dewey’s concept of reflection. She believes 


Reflection is a meaning-making process that moves a learner from one ex- 
perience into the next, with deeper understanding of its relationships with, 
and connections to, other experiences and ideas ... 
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Reflection is a systematic, rigorous and disciplined way of thinking 
Reflection needs to happen in community, in interaction with others 
Reflection requires attitudes that value the personal and intellectual 
growth of oneself and others. 

(Rodgers, 2002) 


In a study by Boud and colleagues, 


Reflection is an important human activity in which people recapture their 
experience, think about it, mull it over and evaluate it. It is this working 
with experience that is important in learning. 

(Boud et al., 1985) 


Sch6n defines 


Reflective practice as the practice by which professionals become aware of 
their implicit knowledge base and learn from their experience. 
(Schon, 1983) 


There are many definitions of reflection, and through many researchers and 
studies, we formulate our reflection idea from numerous definitions as 


Reflection can be generally seen as a process of thinking, evaluating, and 
making sense of existing experiences as well as planning for future expe- 
riences, and is an integral component of both self-knowledge and self- 
regulation allowing individual to evaluate and gain insights into themselves, 
as well as to monitor and change courses of action taken in order to improve 
performance personally and professionally. 

(Chan & Lee, 2021) 


In summary, 


Reflection is how you see yourself before, now and after; how you see 
yourself from different perspectives; how you see yourself after certain 
situations or experiences; how you see yourself, your actions and your be- 
haviours after you observe others. 


5.2 Reflection for Higher-Order Thinking Processes 


Reflection can be complex and challenging, but at the same time, it can be 
highly effective in developing metacognition for higher-order thinking processes 
(Efklides, 2008; Magno, 2010). In Bloom’s taxonomy, reflection and reflective 
practices are considered major approaches to delivering evaluation and critique 
at the highest educational level of the taxonomy (Bloom, 1956). While in Biggs’ 
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Structure of Observed Learning Outcomes (SOLO) taxonomy, John Biggs has 
indicated that reflection maps into deep learning within the SOLO levels; without 
reflection in the teaching and learning activities, surface learning arises (Biggs & 
Collis, 1982). Mezirow (1990) and Moon (1999) found that critical reflection and 
critical review could lead to transformative learning. Transformative learning is a 
process in which learners acquire new information and at the same time, evaluate 
their past ideas and knowledge, and critique what they knew or understood pre- 
viously by combining the new perspectives to derive their own new insights and 
information. Reflection can lead to self-development and self-awareness, research- 
ers (Harvey et al., 2010; Smith et al., 2007) suggested that holistic competencies 
can be developed through reflective practice. Thus, it can be seen that reflection 
is crucial for experiential learning and holistic competency development. 


5.3 The Benefits of Reflection 


Apart from enhancing higher-order thinking competencies, reflection has many 
benefits for teachers and students. Similar to Chapter 1, for simplicity, a compre- 
hensive list of benefits of reflection for teachers and students is presented below. 


Benefits for Teachers 


° Understanding the progress of students — Reflective journals provide good 
opportunities for teachers to gain a better understanding of how their 
students think and feel, as well as the learning progress their students go 
through during the course or activity, which will enhance the students’ 
learning process. 

° Opening constructive dialogues — Reflective journals, if supported by teach- 
ers’ feedback effectively, help open dialogues between the student and the 
teacher to build on past learning experiences and co-construct knowledge. 


Benefits for Students 


° Active learning — The process of reflection encourages the students to take 
the initiative to be active and self-driven; allows individual learner to explore 
concepts and ideas in relation to their thoughts and feelings from different 
perspectives. 

es Responsible for their own learning — Students can become independent 
thinkers through reflective practice and feel empowered to solve various 
problems on their own. 

° Identity construction — Reflective journals allow students to establish nar- 
ratives of themselves, especially when the students are able to maintain a 
trusting relationship with their teachers. When students are encouraged to 
express themselves, individual identities are given a chance to be nurtured. 

° Improve writing skills - Writing reflective journals can involve students in a 
new form of writing that may be new to them. This exposure can lead to 
improvement in students’ writing skills. 
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° Opportunity to express and understand self — Reflective assignments pro- 
vide the platform for students to express what they think and feel about 
the course and their learning process, and also promote their expression 
of ideas, personal experiences and opinions. This is an ideal place for stu- 
dents who are generally not willing to speak up in class and tutorials to 
express themselves. Reflection also enables and assists students to observe 
their own practices, recognise their strengths and weaknesses, and form a 
better self-understanding. 

° Enhancing critical thinking and lifelong learning — The process of self- 
reflection enhances the development of critical thinking skills among stu- 
dents when they relate their knowledge to real-world issues. It can help 
students develop a questioning attitude towards different issues and prob- 
lems and learn to update their learning and knowledge. 

° Keeping track of personal growth — Reflective journals, if logged, provide 
written proof of a student’s progression in thinking. By reviewing these 
journals written over time, students can learn from their growth path and 
strategically plan for their future learning. 


5.4 The Challenges of Reflection 


Reflection as an assessment is exceptionally challenging because it requires the 
assessors to understand more than the knowledge, skills and the activity in- 
volved; often, the assessors need to be aware of learner’s past, present and fu- 
ture directions. In 2021, Katherine (my postgraduate student) and I (Chan & 
Lee, 2021) conducted a comprehensive systematic literature review to discuss the 
challenges of reflections through a multilevel perspective. These challenges are 
classified into four levels as shown in Figure 5.1, which interrelate and influence 
each other in reciprocal relationships. The four levels, from micro to macro in- 
fluences, are (1) student learning level; (2) teacher pedagogical level; (3) institu- 
tional level and (4) sociocultural level. 

In reciprocal relationships, the influences of macro variables (institutional and 
sociocultural) emerge as cultural norms and expectations to create a specific 
social and cultural context that highly influences reflective practice (Boud & 
Walker, 1998), hence determining the appropriate types of reflection for foster- 
ing students’ reflective ability and for institutions to incorporate into curricula, 
in order to engage learners in reflection. Simultaneously, micro variables (student 
learning and teacher pedagogical) also influence higher-level challenges (Erez & 
Gati, 2004). Since education culture is constituted by teachers’ and students’ 
mindsets and practices, changes need to start on an individual level. 


5.4.1 Student Learning Level 


The framework of student learning level is classified into four categories: 
(1) Student motivation, (2) Understanding of and ability to engage in reflection, 
(3) Ethical concerns and emotional impact, and (4) Student-teacher relationship. 
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Figure 5.1 Multilevel challenges in critical reflection (Chan & Lee, 2021). 


Source: Reproduced with permission from authors. 


Student Learning Level: Student Motivation 


To encourage students to engage in reflection, students need to first feel mo- 
tivated. However, we (Chan & Lee, 2021) found that the time and workload 
required for students to develop reflective skills and complete reflective tasks 
hindered students’ motivation (Chong, 2009; Korucu Kis & Kartal, 2019). In 
addition, since reflection is a complex thinking process, pressure increases when 
there are time constraints (Jindal-Snape & Holmes, 2009; Mortari, 2012). Fur- 
thermore, some students did not find the mode of reflection helpful (Ahmed, 
2020; Power, 2012). Neglecting students’ needs and preferences discourages 
them to engage in reflection. 


Student Learning Level: Understanding of and Ability to Engage in 
Reflection 


It is necessary for students to understand what reflection is for and how it is 
relevant to their learning because these affect the quality of student reflection. 
However, from our literature review, Chan and Lee (2021) discovered that some 
students were unable to identify the purpose of reflection. Some also lack the 
necessary knowledge (i.e. reflective theories, subject knowledge, etc.) and rele- 
vant skills (i.e. critical thinking skills, writing skills, etc.) for engaging in reflec- 
tion (Ahmed, 2020; Kis & Kartal, 2019; Thomson et al., 2019). Some failed 
to identify alternative forms of reflection and recognise their values (Jack & II- 
lingworth, 2019). Apart from these challenges, sometimes language barrier also 
obstructs students from engaging in reflection. Unlike other kinds of academic 
writing that are relatively objective, sophisticated rhetoric and linguistic skills are 
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required for reflections (Ono & Ichii, 2019; Rai, 2006). Students’ unfamiliarity 
with the appropriate writing style therefore become a challenge. 


Student Learning Level: Ethical Concerns and Emotional Impact 


It was also found that students might feel stressed, fearful and uncomfortable 
while engaging in deep-level reflections (Ghaye, 2007; Mortari, 2012). Some 
students regarded reflection as private and personal, and hence feel reluctant 
and anxious, or even threatened, to share reflections publicly (Jindal-Snape & 
Holmes, 2009; Ross, 2011). According to Boud (2001, p. 15), 


the exploration of the self that reflection involves requires a relatively 
protected environment in which one is not continually preoccupied by 
defending oneself from the scrutiny of others... For example, revealing 
negative feelings about the difficulties of classroom practice could have a 
substantial influence on how a student teacher is perceived by supervisors 
and may lead to failure to graduate. Whether this is the reality of the situ- 
ation, simply imagining such an occurrence may be a barrier to recording 
such feelings. 


This also further leads to the concerns of ethics, objectivity in grading, and 
the transparency of assessment criteria and process in evaluating personal and 
subjective reflection. Furthermore, students felt unsupported without teacher 
assistance and clear guidelines on reflective task requirements, and this leads to 
anxiety (O’Connell & Dyment, 2005; Sutton et al., 2007). All these discourage 
students from engaging in reflection. 


Student Learning Level: Student-Itacher Relationship 


To encourage students to engage in reflection, students need to feel comfortable 
to share their thoughts and feelings. However, we (Chan & Lee, 2021) found 
that because of teachers’ roles as educator and assessor simultaneously, students 
may not feel comfortable to share deeply (Jindal-Snape & Holmes, 2009), espe- 
cially when sensitive topics were reflected upon. Without sufficient trust between 
students and teachers, students may feel doubtful and anxious about how teach- 
ers assess their reflections (Stewart & Richardson, 2000; Wong-Wylie, 2007). 
Therefore, student-teacher relationship characterised by a lack of trust can ob- 
struct students from engaging in deep-level reflections. 


5.4.2 Teacher Pedagogical Level 


At the teacher pedagogical level, four categories of challenges emerged (Chan & 
Lee, 2021): (1) Difficulties with pedagogical logistics, (2) Conceptualisation of 
terms, purposes and processes, (3) Reflection approaches, and (4) Assessment 
and feedback. 
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Teacher Pedagogical Level: Difficulties with Pedagogical Logistics 


Similar to the challenges faced by students, our literature review paper (Chan & 
Lee, 2021) shows that lack of time was also a challenge for teachers, especially 
when designing reflection for large classes and under a tight teaching schedule. 
Students need to be engaged in deep learning for effective reflection. Therefore, 
teachers could have difficulties in designing a well-structured lesson with reflec- 
tion (i.e. Kuswandono, 2014; Lo, 2010) when there is a lack of time. Accord- 
ingly, time and space resources were also required to support a well-structured 
lesson, but it was found to be difficult for teachers to have access to such resource 
for a large group setting. Teachers also need to possess sensitivity and pedagogi- 
cal competence to facilitate collective reflection safely (Clarke, 2011). 


Teacher Pedagogical Level: Conceptualisation of Terms, Purposes and Processes 


Reflection can be valuable to students if there is meaningful guidance from 
teachers. However, it was found that teachers often lack the necessary knowledge 
to facilitate reflection effectively, and this can prevent students from engaging 
in reflection (Butani et al., 2017; Thomson et al., 2019). In addition, different 
expectations on and understanding of reflection by teachers could also bring 
challenges because the absence of agreement poses difficulties for students to 
understand how they were expected to engage in reflection and how they would 
be assessed (Butani et al., 2017; Lau, 2016). 


Teacher Pedagogical Level: Reflection Approaches 


As discussed earlier, the choice of mode can influence the quality of reflection 
and the motivation of students. Thus, the lack of reflection literacy of teachers 
poses as a barrier for recognising the appropriate modes and approaches suitable 
for facilitating and assessing reflection according to student background and 
expectation (Esposito & Freda, 2016; Power, 2012). 


Teacher Pedagogical Level: Assessment and Feedback 


Apart from challenges in designing reflections, teachers also encounter chal- 
lenges when assessing reflection. Subjectivity is a challenge for teachers while 
setting grading standards for reflections (Bourner, 2003; Smith, 2011). The ab- 
sence of a clear assessment rubric leads to teachers’ reliance on their own per- 
sonal judgment and subjective evaluation to grade reflections (Cheng & Chan, 
2019; Jindal-Snape & Holmes, 2009), from which bias may arise. 


5.4.3 Institutional Level 


At institutional level, the challenges are classified into two categories: (1) Teach- 
ing and learning priorities and (2) Institutional support. 
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Institutional Level: Teaching and Learning Priorities 


Teachers tend to pay less effort in introducing reflective practice because the 
introduction of such an innovative pedagogical approach is not a priority; thus, 
the effort to integrate such an approach into teaching is often left unrecognised. 


Institutional Level: Institutional Support 


Without appropriate institutional support on curriculum structure, teacher 
training, assessment policy and criteria, etc. (Chan et al., 2020; Davis, 2003), it 
is difficult for students and teachers to engage in effective reflection as part of 
their curricula. 


5.4.4 Sociocultural Level 


Sociocultural factors “including norms that govern social interactions, relation- 
ships between students and teachers, and acceptable ways of expressing oneself” 
(Chan & Lee, 2021, pp. 11-12) are challenges for reflection. Traditional educa- 
tional system emphasises on passive learning, rationality and academic outcomes, 
and exam-oriented culture is often more valued (Korucu Kis & Kartal, 2019; Lo, 
2010; Lutz et al., 2017). The power distance between teachers and students in 
traditional educational systems also influences the willingness of both parties to 
engage in critical reflective discussion (Richardson, 2004). All these societal and 
cultural influences can discourage students and teachers to engage in reflection. 


5.5 Moments of Reflection 


As complex as reflection already is, the moment (i.e. when) in which the learner re- 
flects is also of significant importance. It may help to achieve deeper reflection during 
the process. Before reflection, there is Preflection (Jones & Bjelland, 2004), a type of 
reflection to get students thinking about what to expect in their reflection, preparing 
them to reflect; simply put it is the process of thinking about their learning experi- 
ence before it begins. Schón (1983, 1987) advocated two types of reflection, or more 
accurately, moments of reflection — reflection-in-action and reflection-on-action. 


Preflection is a precursor of reflection, it is when one consciously thinks 
about the expectations associated with the reflection. (BEFORE) 


Reflection-in-action occurs at the same time as when the activity (event) 
is happening, the reflection happens simultaneously with the actions, the 
learner thinks and reacts on the spot. (DURING) 


Reflection-on-action occurs after the action, reflecting on something that 
has been done in the activity (event), considering what could have been done 
differently, as well as looking at the positives from that interaction. (AFTER) 


Reflection-for-action occurs when one thinks about future actions with the 
intention to make improvements. (DURING OR AFTER) 
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In education assessment, it is often reflection-on-action we are asking the stu- 
dents to undertake, however, in experiential learning, both types of reflection 
may happen. Another moment of reflection which was developed from Schón's 
work is reflection-for-action, this moment occurs when one thinks about future 
actions with the intention to make improvements (Farrell, 2013). 

Assuming there is a timeline for which reflection should occur, these moments 
of reflection should occur in sequence, 


Reflection- Reflection- 


Reflection- 


Preflection 


on-action for-action 


in-action 


but of course, it is not that simple, reflection does not really happen linearly. 
Our minds often jump from one thought to another, thus, the four moments 
of reflection may occur at any time in the duration of the course. Depending 
on the aim of the reflection, teachers should design the reflection exercise at the 
appropriate moment. 


5.6 Reflection Approaches 


Unlike traditional teaching approaches, reflection is a key component in active 
learning as students actively engage in the process of thinking about what they 
have learnt or experienced. Reflection can be a learning tool and at the same 
time, it can be an assessment (Chan & Luo, 2020). Both modes can achieve 
some or all the benefits discussed in Section 5.3, particularly if it is well designed 
together with effective facilitation of student reflection. 

Mezirow (1991) highlighted how reflection can mean many things, including 
awareness of perception, thought, feeling, intention and action, taking some- 
thing into consideration or simply visualising other options. As such, different 
reflective approaches may allow learners to produce whole or partial aggregation 
of these effects, resulting in the development of different learning outcomes 
including various competencies. This echoes the conclusion of our literature 
review paper (Chan & Lee, 2021), which highlighted that students’ ability to 
“express their thoughts and emotions through different means” (p. 14) is an 
important part of being reflection literate. 

Moon (2004) has long ago suggested that reflection can be represented in 
written, visual, oral or performance forms. Amongst these different forms, 
written reflection has been the most commonly employed approach to nurture 
students’ reflective literacy (e.g. Wald & Reis, 2010). Some of the common writ- 
ten approaches are online journaling (e.g. Xie et al., 2008), diaries (Bruno & 
Dell’Aversana, 2017), and learning portfolios (e.g. Klenowski et al., 2006; Lo, 
2010; Scott & Fortune, 2009), reflective journaling (Barney & Mackinlay, 
2010), reflection essays (McGuire et al., 2009), and reflective analyses (e.g. 
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Table 5.1 Reflective practices under the four-broad type of reflective approaches 


Written Reflection Video Reflection Audio Reflection Face-to-Face Reflection 


Reflective essays Video blogs Podcast Reflective discussion 

Diary (Daily/ Video diary Audio diary Interviews 

Weekly) 

Online reflective Self-recorded Self-recorded Group discussion (with 

journaling video reflection audio reflection mentor/supervisor/ 
peer/teacher) 


Learning portfolio 


Reflective analyses 


Fakazli & Gónen, 2017). With the advent of technology, teachers and research- 
ers have been introducing diverse and innovative approaches to reflection, such 
as video-based reflections (such as vlogs; Rich & Hannafin, 2009), audio-based 
reflection (Myers et al., 2017) and podcasts (Bolliger & Armier, 2013). Other 
reflective approaches include working together with peers on group-based or 
collaborative discussions as well as discussions or interview sessions with super- 
visors (e.g. Burchell & Dyson, 2005; Epler et al., 2013; Martin & Double, 1998; 
McKenna et al., 2009; Olson et al., 2016). This shows the diversity in which 
reflection can be applied in education and how reflection can be an individual 
or group effort. There has also been an increasing interest in multimodality in 
reflection using various representational modes such as images and spoken words 
to encourage reflective practice that can accommodate different learning styles 
and preferences (Barton & Ryan, 2014; Koole et al., 2011; Yuan & Mak, 2018; 
Table 5.1). 

In our recently published paper in the Active Learning in Higher Educa- 
tion journal (Chan & Wong, 2021), we examined student perspectives on the 
four types of different reflective approaches (i.c. written, audio, video and face- 
to-face). We found these different approaches have their own characteristics — 
namely Interactive, Individual and Rework Reflection, and with appropriate 
personal and environmental factors designed into the experiential learning pro- 
gramme, holistic competency can be developed. Figure 5.2 shows the interac- 
tive, individual and rework reflection framework, and more information can be 
found in Chan & Wong’s 2021 paper. 

Although written reflection is the most common type of reflection practised as 
an individual reflection, students predominantly preferred interactive reflection 
when given a choice. Face-to-face reflection should be increasingly implemented 
to encourage learning as Rodgers (2002) has long stated that “reflection needs 
to happen in community, in interaction with others." However, to implement 
effective interactive reflection, it is vital to build trust and good relationships 
among the learners and the respective stakeholders whether it is the mentor, 
supervisor, peer or teacher. 
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Interactive Reflection 


* Reflection is shared 

* Natural 

* Immediate response 

*Provide appropriate feedback 

e Reflexivity in reflection 

Relationship and trust 

e Approach example: face to face reflection 


Individual Reflection 


e Reflection is private 

* Deeper thinking process 

* Suitable for those who are not confident to share or 
express in group 

* Adept writing may cause biased 

* Approach example: written reflection 


Rework Reflection 


Reflection can be private and shared 

*Reflection can be further reflected - "second reflection 
action" 

» Reflexivity in reflection 

*Nuture self-responsible in reflection 

Think aloud to observe errors 
Approach example: video or audio reflection 


Figure 5.2 Interactive, individual and rework reflection framework (Chan & Wong, 2021). 


Source: Reproduced with permission from authors. 


In the literature, reflection has always been assumed as an individual, private 
journey. Words associated with reflection such as our thoughts, our expe- 
riences, our feelings and our emotions are often intertwined with personal 
diary, individual journal and self-reflective writing, emphasising reflection 
should be secluded and isolated. This study has shown reflective approach 
such as face-to-face can be interactive and communal if the learning envi- 


ronment is built properly. 
(Chan & Wong, 2021) 


When designing reflective practices, it is important to focus on students’ needs. 
Students deeply care about their privacy, and sometimes, “forced reflection” re- 
corded either via video or audio or via face-to-face individual or group interviews 
may not be comfortable for them (Hobbs, 2007). One suggestion is to allow 
students to choose their choice of approaches to reflect, as this will foster learner 
autonomy and learner responsibility. The drawing in Figure 5.3 by Grace Collo- 
ton demonstrates the different reflection types. 

In the findings, audio and video reflection are categorised into rework 
reflection, 


Rework reflection is defined as the type of reflection approach which al- 
lows students the opportunity to perform “at least” a “second reflection 
action”, allowing the students to rework on their reflection from their initial 
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Figure 5.3 Reflection types — individual, interactive and rework reflection (Drawing by 
Grace Colloton). 
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reflection. The rework reflection may be from their own reflection or from 
reflecting on peers’ reflection. 
(Chan & Wong, 2021) 


Learners are not accustomed to this kind of reflection, so there is a need to 
not only promote varying reflective practices but also to equip teachers with 
the knowledge and competence to implement such practices and to ensure that 
teachers are assessment literate in reflection (Chan & Lee, 2021; Chan & Luo, 
2020; Chan et al., 2020; Cheng & Chan, 2019), while taking into consideration 
students’ needs and preferences. 


5.7 Reflection Assessment Frameworks and Models in 
Higher Education 


With a variety of reflection concepts and approaches, there are also a variety of 
frameworks and models for investigating and guiding reflective practice within 
different disciplines in higher education. In the following section, I will provide 
a review of seven prominent frameworks and models that are utilised for the 
assessment of student reflection in higher education. 


5.7.1 Kember et al. Coding Scheme and Mezirow’s Typology of 
Reflective and Non-Reflective Action 


Mezirow’s (1991) typology of reflective and non-reflective action has been 
widely adopted and adapted to measure students’ reflective thinking in different 
disciplinary areas. In his typology, there are three dimensions: content reflection 
(reflection on what we perceive, think, and act), process reflection (reflection 
on how we perceive, think, and act), and premise reflection (becoming aware 
of our own ways of perceiving, thinking, and acting). He further distinguished 
reflective action from non-reflective action. Non-reflective action comprises ha- 
bitual action, thoughtful action in which previous knowledge may be recalled 
but not critically assessed, and introspection which is purely thoughts about 
ourselves without further analysis. Reflective action, on the other hand, con- 
cerns the three dimensions of content, process, and premise reflection. Kember 
et al. (1999) devised a coding scheme based on Mezirow’s notions of reflective 
and non-reflective action. They reordered the seven types of thinking within 
the typology into four levels as shown in Table 5.2. Kember, Wong, and Leung 
(1999) argued that content reflection and process reflection are at the same level 
of complexity, while premise reflection is at a higher level of reflective practice. 
In line with Mezirow’s conceptualisation, the first two levels were categorised as 
non-reflective whereas the last two levels were grouped under reflective action. A 
new category was constructed to account for the co-occurrence of content reflec- 
tion and process reflection in journal writing. Eight assessors tested the coding 
scheme on three journal entries written by three undergraduate healthcare 
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Table 5.2 Kember et al. (1999) coding categories for reflective assessment based on 
Mezirow’s typology 


Level | Mezirow’s (1991) Typology of Reflective and Kember Kember et al. 
Non-reflective Action et al. (1999) | (2008) 
4 Premise reflection Reflective Critical 
action reflection 
3 Content Process Content Reflection 
reflection reflection and 
process 
reflection 
2 Introspection Thoughtful action Non- Understanding 
] Habitual action EE Non-reflection 
Action 


students, and the results showed that the coding scheme was a valid instrument 
for judging the quality of the research participants’ reflective journals. 

Bell et al. (2011) trialled Kember et al. (1999) coding scheme on seven business 
education students’ written journals and proposed two additional categories— 
Process Reflection (Internal) and Process Reflection (Others) to differentiate 
students’ reflection on their own thinking processes and those of others. They 
found that the coding scheme was useful for identifying the levels of reflection 
attained by students, but it was too detailed for coding a large number of written 
journals. 

Kember and colleagues later reformulated the coding scheme as an assess- 
ment protocol with four categories of reflection (Kember et al., 2008). In the 
revised four-category scheme, habitual action is retained as the first level of non- 
reflection. The other three categories are understanding, reflection, and critical 
reflection at the second, third, and fourth levels respectively. The protocol is 
intended for judging a piece of writing as a whole by assigning the highest level 
of thinking observed in the entire piece. In some of the studies that adapted 
Mezirow’s typology, students were graded as non-reflectors, reflectors, or criti- 
cal reflectors based on the reflective elements observed in their written journals 
(Chirema, 2007; Paton, 2006). 


5.7.2 Van Manen’s Three Levels of Reflectivity 


Van Manen (1977) outlined three levels of reflectivity based on theories in so- 
cial sciences and education. The first level, technical rationality, concerns the 
technical application of knowledge and concepts to achieve a given goal without 
taking into consideration the wider context. On the second level, practical ac- 
tion, ideas and underlying assumptions are clarified and analysed to evaluate the 
consequences of action. The focus is on the nature and quality of an experience 
and the practical actions taken. The highest level is critical reflection where social 
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Table 5.3 Van Manen's three levels of reflectivity 


Level Reflective Levels Brief Criteria 

3 Critical reflection Take into consideration personal biases, issues of 
power 

2 Practical action Focus on the nature and quality of experience and 


action taken 
1 Technical rationality Concerns about technical application of knowledge 
and concepts only 


circumstances are taken into consideration, personal biases are questioned, and 
issues of power and domination are critically evaluated (Table 5.3). 

Min et al. (2016) compared the reflective thinking of seven student teach- 
ers who had not been exposed to the critical reflection manual and that of 11 
student teachers who had the exposure, using Van Manen’s model to analyse 
their self-reflection notes and weekly journal entries. Their results show that the 
reflective writing of 94.4% of the participants who had not been exposed to the 
critical reflection manual was at the technical level. In contrast, among the 11 
student teachers who had been given guidance on the use of the critical reflec- 
tion manual, 81.3% of them could reflect critically, and none of them displayed 
technical rationality. In another study, Van Manen's model was used to analyse 
the interview data and written assignments of five physical education preservice 
teachers (Ballard & McBride, 2010). The researchers noted an increase in the 
participants’ reflectivity over the 12-week period of data collection, and they 
concluded that Van Manen's model was a valid instrument for tracking the de- 
velopment of reflective thinking in teachers. 


5.7.3 Bradley’s Three Levels of Reflection 


Bradley's three levels of reflection (as cited in Jacoby, 2015) provide an assessment 
framework that outlines the criteria for judging the level of student reflection. At 
the first level, reflection is superficial, one-dimensional, and narrow in its scope. 
It focuses on one aspect of the event and tends to be descriptive rather than 
analytical. At the second level, there is emerging evidence of critical reflection 
but with limited depth. Students may attempt to make connections, interpret 
evidence, and draw conclusions. They may also be able to make more thorough 
Observations of the situation than level one, but they still fail to position the 
Observations within a broader context and consider multiple perspectives. To at- 
tain the third level, students must demonstrate the ability to examine a situation 
within a broader and more complex context. There should be evidence of logical 
reasoning, well-grounded judgements, and consideration of multiple viewpoints. 
Bradley's assessment criteria have been employed to analyse teacher reflection in 
digital stories (Walters et al., 2011) and to assess student reflection as the out- 
come of experiential learning (Straughan et al., 2018). 
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5.74 King and Kitchener’s Reflective Judgement Model 


In King and Kitchener’s (1994) reflective judgement model, reflection is con- 
ceptualised as seven stages progressing from pre-reflective thinking (stages one, 
two, and three) to quasi-reflective thinking (stages four and five) and reflective 
thinking (stages six and seven). The level of reflection is described in terms of 
views of knowledge and quality of judgements (King & Kitchener, 2004). Pre- 
reflective thinking is characterised by a perception of knowledge as absolute with 
little or no justification given to support one’s views. As students progress to the 
phase of quasi-reflective thinking, they begin to acknowledge that knowledge is 
subjective and that they need to consider other perspectives to interpret evidence 
and events. At the highest level, knowledge is viewed as the product of one’s per- 
sonal construction based on information from multiple sources. Perceptions and 
beliefs are justified by weighing evidence and various viewpoints through logical 
reasoning. The reflective judgement model provides a framework for understand- 
ing the development of reflective thinking in young adults and mature adults 
(King & Kitchener, 2004), and thus it is appropriate for assessing student reflec- 
tion in higher education. For example, Hashemi (2016) employed the model in 
an investigation of graduate students’ intellectual development in a university, 
and Badger (2010) designed questions based on the levels in the model to exam- 
ine preservice teachers’ reflective thinking ability in an oral examination. 


5.7.5 Sparks-Langer, Simmons, Pasch, Colton, and Starko’s 
Framework for Reflective Thinking 


Another assessment framework was devised by Sparks-Langer et al. (1990) to meas- 
ure teacher education students’ ability to think reflectively on their pedagogical 
decisions. The framework classifies seven levels of reflective thinking based on the 
language used by students. Level one is assigned when no descriptive language is ob- 
served. Level two shows simple, layperson description, and level three is indicated by 
events being labelled with appropriate terms. At levels four and five, explanations are 
supported by personal preference or theory as the rationale, whereas levels six and 
seven are characterised by explanations that take into consideration broader contexts 
and more complex factors such as ethical, moral, and political issues (Table 5.4). 


Table 5.4 Sparks-Langer et al. (1990) reflective pedagogical thinking framework 
Level Brief Criteria 


Explanation of ethical, moral and political issues 
Explanation with broader contexts 

Explanation supported by theory as rationale 
Explanation supported by personal preference as rationale 
Events labelled with appropriate terms 

Simple layperson description 

No descriptive language 


= iS Q + TAN 
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Using the seven levels in Sparks-Langer et al.’s framework, Calandra et al. 
(2008) coded the written reflections of ten physical education teacher candidates 
before and after three learning cycles. They noticed a general increase in the 
participants’ reflective thinking ability as indicated by the levels of reflection 
they attained according to the assessment framework. In another study, Calan- 
dra et al. (2009) used Sparks-Langer et al.’s framework and Van Manen’s (1977) 
levels of reflectivity to analyse the journal entries of six teacher candidates to 
investigate the effects of video editing experiences on the teacher candidates’ 
reflective thinking ability. Several other studies have also reported the use of the 
seven-level framework to investigate teacher reflection such as Sung et al. (2009) 
investigation of teachers’ reflections in digital teaching portfolios and Hussein’s 
(2007) research on teachers’ reflective thoughts through the analysis of 11 pro- 
spective teachers’ written journals. 


5.7.6 Hatton and Smith’s Four Types of Writing 


In the assessment of written reflections, Hatton and Smith’s (1995) categorisa- 
tion of writing is a commonly used framework. Four types of writing were pro- 
posed: descriptive writing, descriptive reflection, dialogic reflection, and critical 
reflection. Descriptive writing is an entry-level, non-reflective account of events 
without any reasons and justifications provided to explain the events. Descrip- 
tive reflection integrates some form of justifications for events, but the writing 
is still mainly descriptive. Although there may be some mention of multiple 
factors that influence one’s experience of events, they are not explored in depth. 
The third level of writing is dialogic reflection which shows the writer taking a 
step back from the events, exploring alternatives, making reasoned judgements, 
and providing rationales for actions. The highest form of writing is critical re- 
flection in which the writer displays an awareness of how wider historical and 
social-political contexts influence events, actions, and the perspectives adopted 
by different people. In Penso et al. (2001) study, this categorisation was used to 
examine novice teachers’ ability to reflect on their practical experiences. Despite 
its origin in teacher education, Hatton and Smith’s framework has also been 
used in research into clinical practitioners’ reflective thinking (Greenfield et al., 
2015; Taylor-Haslip, 2010). 


5.7.7 Moon’s Generic Framework for Reflective Writing 


Moon (2004) built on the work of Hatton and Smith (1995) and proposed a 
four-part generic framework for reflective writing. The first two levels of her 
framework, descriptive writing and descriptive account with some reflection, 
mirror Hatton and Smith’s descriptive writing and descriptive reflection. Moon 
named the third and fourth levels “reflective writing (1)” and “reflective writing 
(2)” respectively. Similar to Hatton and Smith’s (1995) dialogic reflection, re- 
flective writing (1) demonstrates evidence of analysis, evaluation of events, and 
justification for actions. In reflective writing (2), the writer recognises alternative 
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perspectives, acknowledges the uncertain nature of people’s frame of reference, 
and questions the views and motives of self and others. To investigate the rela- 
tionship between the depth of students’ written reflections and their information 
literacy development, McKinney and Sen (2012) graded their research partici- 
pants’ reflective statements using Moon’s four categories of writing. They con- 
cluded that Moon’s framework was a useful tool for assessing students’ learning 
outcomes and for analysing the depth of reflection in their written assessments. 


5.8 Reflection Literacy 


Although reflection is a learning method commonly employed in many educa- 
tional settings at different educational levels, there is still a lack of studies on the 
ability of teachers in higher education in facilitating and assessing reflections, 
or what we referred to in our paper (Chan & Lee, 2021) as “reflection literacy”. 
We outlined the four components of reflection literacy in accordance with the 
research findings in a reflection literacy framework as shown in Figure 5.1. The 
equal status and reciprocal relationship of the four components illustrated by 
the framework suggest that in order to have a comprehensive understanding of 
reflection literacy, concerns of individual’s abilities (i.e. both students and teach- 
ers) in the micro level as well as the institutional and societal contexts in the 
macro level are both necessary. 

While the discussion of reflection literacy in higher education is still gathering 
interest from academics, the following sections may offer a brief overview of 
some recently published contributions from academics on the topic. 

For reflection literacy to be established in higher education, there is a need 
to first overcome some barriers. These challenges are shown in Section 5.4. As 
shown in Figure 5.3, the four components that constitute reflection literacy are 
interrelated. Therefore, efforts must be made in coordinating the implemen- 
tation of reflection in the different challenge levels namely, student learning, 
teacher pedagogical, institutional and socio-cultural levels. With adjustments 
and collaboration between different levels, the challenges identified in the four 
levels can then be overcome and reflection literacy can be further encouraged in 
higher education (Figure 5.4). 

Narrowing down the focus to Hong Kong, the development of reflection lit- 
eracy in higher education is still in its early stage with various needs remain to 
be addressed. In a study conducted by my student Jiahui (Jess) and I (Chan & 
Luo, 2020), six university teachers in Hong Kong were recruited to analyse their 
assessment literacy in reflection. The result revealed that despite the robust con- 
ceptual knowledge of reflection that teachers had, a stark gap between theory 
and practice remained as they showed a lack of confidence in assessing reflection. 
In another perspective, entrenched beliefs such as academic-oriented culture and 
the “teachers as the authority” mindset also confined the development of reflec- 
tion literacy for teachers. 

The misapprehension of reflection as an assessment is mind-boggling for both 
students and teachers. On one hand, students are accustomed to present their 
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Figure 5.4 Reflection literacy framework (Chan & Lee, 2021). 


Source: Reproduced with permission from authors. 


best work for assessment, but on the other hand, reflection is not built for show- 
casing students’ best work. This mentality may often contradict the process and 
the outcomes of reflection. Similarly, teachers struggle with the tension of using 
reflection as an assessment (Boud, 2001). To be reflection literate, they need to 
“understand the clear separation of writing for learning and writing for assess- 
ment purposes” (Boud, 2001, p. 16). 

With these needs left unmet, more efforts are still to be made by different 
stakeholders from multiple aspects so that the development of reflection literacy 
in higher education can be further promoted. 


5.9 Tips for Designing Effective Reflective Assessment 
Practice 


As discussed before, reflection as an assessment is exceptionally challenging be- 
cause of the need for assessors to understand more than the knowledge, skills 
and the activity involved, meaning the assessors need to be aware of learner’s 
past experience, present situation and future directions. Some researchers believe 
only active, conscious and systematic reflection count as reflection (Gondo & 
Amis, 2013; Valli, 1997), which, strictly speaking, is correct. However, if the 
reflection is employed as an assessment, then teachers will need to fully embrace 
student development, and encourage learners to intentionally reflect on their 
experiences. Hence, the reflective practice needs to be designed together with 
the activity — it cannot be an after-thought. Below are some tips for designing 
effective reflective assessment practice. 


180 Reflection as Assessment 
Build Habits to Critically Reflect at Appropriate Moments 


Many students either do not have the time or do not know how to critically 
reflect. Providing class time to teach students to uncover and correct their own 
mistakes can be a starting point for reflection. Sample questions for a student 
includes: What went wrong? 

As mentioned in Section 5.5, there are different moments of reflection, and 
reflecting at the appropriate moments can be very effective to enhance student 
learning. When teacher is designing an experiential learning activity, they should 
contemplate the appropriate moment of reflection for students to reflect. Pur- 
posefully designing pockets of reflection practices will result in building students’ 
reflective habits and will assist them to reflect more intentionally and critically. 


Offer Adequate Training and Guidelines 


Reflective approaches are unlike traditional learning activity or assessment, as stu- 
dents may not be familiar with the expectations and deliverables. Thus, like all 
assessment practices, teachers should provide guidelines (Levett-Jones, 2007), ex- 
emplars, practices and trainings (Dyment & O’Connell, 2010) to ease their anxiety 
and uncertainty in participating in reflection (Sutton et al., 2007). 


Avoid Judging Personal Reflection 


Reflection is difficult to assess due to its innate subjective nature. As they of- 
ten result from personal judgements or views based on the learner’s experience, 
given that each learner has their own prior experience and personality (Bourner, 
2003; Butani et al., 2017; Fernandez-Pefia et al., 2016; Smith, 2011), it is in- 
appropriate and even offensive for teachers to directly assess the personal part 
of the reflection. The reflection should be seen as an aid to learners in working 
through a process, not as a product. Rather than assessing the reflective piece 
directly, for example, a reflective journal which students are expected to record 
their reflection including their feelings, experience, judgement, critique, knowl- 
edge or competencies gained in the journal. Teacher can separately design an es- 
say assignment in which students can selectively draw on evidences or sources of 
references from the journal to support arguments to produce a summative piece 
of work. Students should be given the choice to submit the reflective journal or 
not as part of their evidence. Furthermore, Koole et al. (2011) also affirmed that 
direct assessment of the reflective piece will fail to achieve a valid and consistent 
assessment, *because they (teachers) have to judge selective descriptions without 
being able to verify their adequacy" (p. 5). 


Usefulness/Uselessness of Reflection as an Assessment 


Reflection as an assessment is a debatable question. I think that I need to reiter- 
ate my personal reflection from Chapter 3 here *to assess or not to assess, that 
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is the question”, modified from Hamlet’s soliloquy by William Shakespeare. It 
is unhelpful to assign reflection as an assessment in a context in which it is not 
offering learners a useful result (Boud & Walker, 1998, p. 198). 


Be Mindful of the Environment 


It is also important for teachers to be mindful of the context and the environ- 
ment. The interconnected nature of social categorisations such as race, class, 
and gender as they apply to a given individual or group in reflection may cause 
serious dilemmas for the student and raises concern in ethics. Boud (2001, p. 15) 
asserted that 


the exploration of the self that reflection involves requires a relatively pro- 
tected environment in which one is not continually preoccupied by de- 
fending oneself from the scrutiny of others... simply imagining such an 
occurrence may be a barrier to recording such feelings. 


In Chapter 7, more on ethics in the assessment of experiential learning will be 
presented. 


Set Standards and Guidelines 


Most teachers do not have high reflective literacy, thus, without clear guidelines 
or rubrics for assessing reflection, teachers may rely on a random variety of criteria 
to assess students, including word count, grammar and writing comprehensibil- 
ity, evidence of deep reflection, personal judgement and measurement against 
previous cases of assessed work (Cheng & Chan, 2019). The reflection assessment 
frameworks in Section 5.7 and some of the assessment approaches in Chapter 4 
provide examples to guide teachers to develop their rubrics and guidelines. 


Assess Student Progress 


When assessing students’ learning progress, teachers need to gauge students’ 
progress for a long period (Bruno & Dell’Aversana, 2017; Hubbs & Brand, 2010; 
Threlfall, 2014), as without sound understanding of students’ past experiences, it 
is difficult to provide evidence of improvement. Tracking student progress against 
the standards helps students identify the gaps between the expected standards 
and their learning. An approach to assess students’ progress is by focussing on 
how well the students performed after mid-progress feedback and comments. 


Design Interaction Reflection 


Students have indicated that when a trusting and respectful environment with 
peers and teachers is established, interactive reflection (such as face-to-face reflec- 
tion) can be more effective and welcomed (Chan & Wong, 2021). As students 
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share their reflections with one another, they can identify misconceptions or ar- 
eas of confusion together and apply their knowledge and experience to solutions 
in different contexts and disciplines. Nicely put by the saying “two heads are 
better than one,” this mode of reflection may help them to think more deeply 
about their assumptions and beliefs. 


Promote and Allow Students to Choose their Choice of Reflective 
Approaches 


Reflection is often “infused with emotion” (Dean et al., 2012, p. 111), dictating 
students to reflect on a set approach may cause discomfort. 


Be Aware of Unexpected Learning Outcomes in Reflection 


Reflection is emergent. Similar to experiential learning activities, outcomes may 
be undefined in advance and often be unplanned and unexpected. A two-part 
learning contract could be implemented in this case. Student and teacher could 
agree on the assessment and reflective approach for the generic outcomes of 
the experiential learning course and for those unplanned “specific outcomes” 
(Luk & Chan, 2021), students should be given autonomy to design the appropri- 
ate assessment. The first part of the learning contract may be executed at the be- 
ginning of the learning activity to raise awareness, build expectations and agree 
on some basic responsibilities; while the second part of the learning contract can 
be used to manage the unplanned learning outcomes. This can be executed after 
the student understands their position and duties of the activity. 


Act as a Role Model by Being a Reflective Teacher 


Teachers should model expectations by critically reflecting on themselves and 
their teaching. This may assist teachers to adopt less familiar roles, as facilitators 
(Rarieya, 2005) and reflective practitioners (Smith, 2011). 


Provide Feedback by a Qualified Individual 


Feedback should only be provided by qualified individuals who are fit to com- 
ment, and that sometimes may not be the teacher. 


Present Reflective Prompts 


Reflective prompts allow us to direct students towards the expected outcomes of 
a reflective assessment. Teachers could provide reflective questions and prompts 
to link and construct meaning from students’ experiential learning experiences. 
Rolfe et al. (2001) developed three simple but major questions (i.e. What, So 
What and Now What) for reflective practice. With these three “what” questions, 
I have generated a list of prompt questions that would be useful in helping stu- 
dents reflect as shown in Table 5.5. 
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Table 5.5 The three what questions and reflective prompts 


The WHAT Question and Description 


Reflective Prompt Questions 


What? 


This section is for the student to write a 
description of the experience with concepts 
and brief details. 


So what? 

This section is for the student to derive 
more in-depth details of the experience 
including your different viewpoints, 
actions, understanding, emotions, etc. 
Using the information from the “what?” 
section as well as your previous experience 
and knowledge to help you to think 
through the different perspectives. 


Now what? 

This section is to explore how the 
emotions, actions, viewpoints and impacts 
that result from the experience will shape 
future thinking, behaviour and action and 
develop learning. 


Conclusions 


What happened? 

What did you expect? 

What were your role in the project? 
What did you observe? 

What was your reaction? 

What did you learn? 

Who was involved in the project? 
When did this project occur? 
Where was the project based? 
Why did you pursue this project? 
How did you feel after the project? 


How did the experience enhance your 
personal development? 

Did you identify strengths or weaknesses 
while participating this project? 

What kind of impacts did the project 
achieve and how did it impact you? 
Were there any discrepancies between 
your expectations and the outcomes? 
What surprises you and why? 

What did you learn about others? 
What did you learn about yourself? 
Where could I have done better? 


What will you do differently in the 
future? 

How would you achieve your goals 
based on your experience? 

What do you need to do to overcome 
the challenges in the future? 

How will you apply what you have 
developed from this experience? 
How can I do better next time? 


In this chapter, I presented topics on reflection as assessment in experiential 
learning. Reflection is a fascinating area, it is absolutely necessary if we want stu- 
dents to develop and improve. However, whether it should be used as an assess- 
ment, I am still debating it internally because unless teachers and students are 
reflection and assessment literate, they may find it difficult to fully unite the two 
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together. Nevertheless, I am certain the different sections in this chapter will 
increase one’s reflection literacy, and hopefully, this will help us move towards a 
more “fair, gifted, innovative” approach in assessing reflection. 


Questions to Ponder 


e Should you use reflection as a learning activity or as an assessment? 
e When is the right moment to reflect? 

e What kind of learning outcomes can reflection achieve? 

e How to avoid judging personal reflection? 

e Isit ethically correct to assess students’ reflection? 

e How can habits be built for intentional reflection? 


Personal Reflection 


The morning after I finished writing this chapter. I lay on my bed think- 
ing, “have I missed anything?” “Are all sections interconnected in a co- 
herent way?? *Would experts agree with my take on reflection literacy?" 

I guess I am practicing what I preach, trying to critically and intention- 
ally reflect. 

Whether reflection should be assessed or not, I think this is something 
the teacher needs to design with caution, requiring reflection as an assess- 
ment may produce contrary and unfavourable effects. Reflection literacy is 
needed for both students and teachers, and no teacher should use reflec- 
tion without being fully literate on it. 

Ifyou would like to learn more, connect with me on Twitter and Linke- 
dIn, as I plan to run workshops for teachers and also guide them to teach 
students how to reflect. I hope I can share what I have learnt with you all. 
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6 Feedback in 
Experiential Learning 


Feedback can be emotional; it is just a fine line between critiquing and criticising, 
depending what you are hoping to achieve. 
— Chan, CKY 


Introduction 


In Chapter 3, I introduced reflection and feedback as part of the outcome-based 
approach for experiential learning, as these two components are particularly vital 
in its development. Reflection has already been discussed in detail in Chapter 5, 
and in this chapter, we will examine feedback in experiential learning and pro- 
vide some strategies with which feedback can be designed in an experiential 
learning curriculum. However, feedback is a topical and very complex area, and 
a vast amount of energy has been exhausted by many great colleagues such as 
David Boud, David Carless, Gavin Brown, Joanna Tai, Phillip Dawson, David 
Nicol, Rola Ajjawi, John Hattie, Naomi Winstone, Yan Zi, Ivan S W Chong, 
Christopher Deneen, Debra Macfarlane-Dick, Royce Sadler, Emma Medland, 
and many more (forgive me if your name is not mentioned here... there are 
just so many fantastic researchers on feedback). Thus, I only include this brief 
chapter on feedback in experiential learning to make this book more complete. 
If feedback in experiential learning is what interests you, I invite you to take 
a look at the articles by these colleagues and try to adapt their strategies into 
experiential learning. A reference list of their articles can be found at the end of 
this chapter. 


6.1 Feedback is ... 


The meaning of feedback has embraced significant conceptual shifts over the 
recent decades. Traditionally, feedback has been defined as information deliv- 
ered to students, with a focus on how teachers engage with feedback (Hattie & 
Timperley, 2007). However, more recent understandings go beyond seeing feed- 
back merely as a unidirectional transmission, but as “a process through which 
learners make sense of information from various sources and use it to enhance 
their work or learning strategies” (Carless & Boud, 2018, p. 1315). Following 
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Carless and Boud’s model, the role of students has been emphasised in the feed- 
back process. Some latest research on feedback operates from ecological (Chong, 
2021) and socio-material perspectives (Gravett, 2020), foregrounding individ- 
ual, contextual, spatial and temporal factors that influence meaningful student 
engagement with feedback. 

Along with reflection, feedback is the most critical element of effective expe- 
riential learning programmes (Eyler, 2009). To elucidate the theory underlying 
experiential learning, Kolb and Kolb (2005) propose that an effective higher 
education programme needs to emphasise the process of learning — one which 
integrates feedback on students’ various learning attempts and efforts. As Ryser, 
Halseth, and Thien (2009) put it, *[w]ithout sufficient guidance and feedback, 
experiential learning cannot take place” (pp. 256—257). When carefully planned 
and well executed, feedback can determine task performance (Bhattacharya & 
Neelam, 2018), provide a basis for self-reflection (Quinton & Smallbone, 2010), 
and enhance learning transfer (Furman & Sibthorp, 2013). In what follows, I 
review some common focuses in feedback research and relate these focuses to the 
context of experiential learning in higher education. 


6.2 Components in the Feedback Mechanism of 
Experiential Learning in Higher Education 


Under the traditional framing of feedback as information, research has sug- 
gested several elements critical to feedback input in experiential learning. For 
instance, feedback should be individualised (Jerman, 2002; Parikh et al., 2001; 
Perera et al., 2008), which is particularly important for holistic competencies, 
as they are often personal; feedback also needs to be specific and detailed (Pel- 
grim et al., 2012; White, 2007), and provided in a timely manner (Perera et al., 
2008). Following the shift in reconceptualising feedback as a student-centred 
process, increasing research has also argued to cultivate students’ feedback liter- 
acy as a way of enabling their uptake of feedback (Carless & Boud, 2018; Han & 
Xu, 2020). However, despite increasing investment and efforts to improve ex- 
periential learning feedback practices (e.g., Dannefer & Prayson, 2013; Kamp 
et al., 2014), dissatisfaction among teachers and students regarding feedback as 
a whole still remains fairly palpable in higher education worldwide (Henderson 
et al., 2019). 

A review of relevant literature shows that the feedback mechanism of experien- 
tial learning in higher education is made up of the following four components. 
The first three components are frequently discussed and considered important 
under the traditional paradigm of feedback; whereas the last one, which focuses 
on the interaction between teacher and student feedback literacy, reflects some 
recent efforts to reframe feedback as a student-oriented process. The four com- 
ponents to be discussed are: 


i Quality of feedback input 
ii Qualified sources of feedback 
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ii Types of feedback practices 
iv Student and teacher feedback literacy 


6.2.1 Quality of Feedback Input 


Many studies have looked into what constitutes ‘good’ or ‘effective’ feedback 
input (e.g., Dawson et al., 2019; Ferguson, 2011; Nicol & Macfarlane-Dick, 
2006). To date, one of the most well-cited research on this topic was authored 
by Nicol and Macfarlane-Dick (2006), which has been cited over 6,000 times 
as of 2022. Based on the seven principles they proposed (pp. 206-214), quality 
feedback input: 


i Helps clarify what good performance is; 
ii Facilitates the development of self-assessment (reflection) in learning; 
ii Delivers high-quality information to students about their learning; 
iv Encourages teacher and peer dialogue around learning; 
v Encourages positive motivational beliefs and self-esteem; 
vi Provides opportunities to close the gap between current and desired 
performance; 
vii Provides information to teachers that can be used to help shape the teaching 


Apart from these principles, some other features of feedback input, such as be- 
ing detailed, personal, timely, specific and criteria-referenced, have also been 
frequently discussed in the literature (e.g., Dawson et al., 2019; Ferguson, 2011; 
Poulos & Mahony 2008). Due to space limitations, here I only elaborate on two 
widely recognised feedback features that are particularly relevant to the experi- 
ential learning context — timeliness and specificity. 


Timeliness 


Feedback in experiential learning should be timely for it to be perceived as use- 
ful and actionable (Powell & Bartlett, 2018; Watling, 2014). Timeliness does 
not necessarily mean promptness but rather the fact that feedback is given at a 
suitable time to enhance learning. It is particularly vital in industrial internships 
to ensure close monitoring, as any mistakes will potentially jeopardise clients’ 
interests (Bhattacharya & Neelam, 2018). In an investigation into the effective- 
ness of an e-business project, Daly (2001) found that a lack of prompt feedback 
from the instructors resulted in a sense of confusion and ambiguity among the 
students about the direction of their work and learning objectives. When prop- 
erly implemented, concurrent feedback alleviates learner anxiety and promotes 
self-efficacy (Daly, 2001). As engaging teachers in experiential learning designs 
already places a heavier workload on them compared with adhering to tradi- 
tional pedagogies (Li et al., 2007; Ngai & Chan, 2019), teachers might not have 
the capacity to turnaround feedback timely or in detail for a large class (Garcia 
et al., 2017). 
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Specificity 


The interactive and authentic features of experiential learning call for novel 
assessment methods to evaluate students’ learning. As mentioned in Chap- 
ter 4, reflective writings (Deeley, 2018), portfolios (Brown, 2002; Qualters, 
2010), observations (Cheng et al., 2019; Ferguson et al., 2016) and many 
other assessment methods have already been widely adopted in experiential 
learning. Due to the subjective nature of these methods and commonplace of 
unintended learning outcomes, teachers reported difficulties in providing pre- 
cise and constructive feedback to students (Lean et al., 2017; Ramsgaard & 
Christensen, 2018). Teachers also have to deal with the socio-emotional as- 
pect of feedback. Bradford (2019) asserted that inappropriate feedback of stu- 
dents’ experiential learning could cause “pain and damaged relationships” 


(p. 95). 


6.2.2 Qualified Sources of Feedback 


One of the key issues in implementing feedback within the experiential learning 
context concerns who should provide feedback and whether they are qualified 
sources of feedback. The word “qualified” can mean trusted, experienced and/ 
or respected. Students may question the reliability of the feedback particularly 
when the feedback is not associated with the academic-related outcomes of the 
experiential learning project, but rather with holistic competency. Depending on 
the type of experiential learning activity, other sources of feedback may be more 
suitable, such as peers, external partners, seniors and self. 


Peer Feedback 


Research has revealed evidence that peer mentoring and feedback produce pos- 
itive outcomes across social, emotional, behavioural, and academic domains for 
adolescents when implemented effectively (e.g., DuBois et al., 2011; Goldner & 
Mayseless, 2009; Raposa et al., 2019; Rhodes, 2002), as adolescents often found 
it easier to associate with their peers. In experiential learning, as noted by Van- 
Schenkhof et al. (2018), peer feedback is a critical component and is “directly 
associated with problem-based, team-based, and classroom organization based 
instructional methods” (p. 92). Carvalho (2013) also stated that peer evaluation 
has attracted ample attention in higher education, particularly in regard to ex- 
periential learning and active learning. Parikh et al. (2001) surveyed 103 medi- 
cal students and found that individual feedback from tutors alongside peer and 
group feedback is most useful to students’ problem-based learning. In one of my 
holistic competency development programmes, a near-peer mentoring scheme 
was organised for over 500 secondary schools, where the participating mentors 
were all university students or fresh graduates in Hong Kong (Chan & Luo, 
2020, 2021a). Informed by findings from both mentors’ interviews and mentees’ 
written/video reflections, six effective and impactful mentoring strategies were 
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devised. These are (i). Creating opportunities to explore, act and reflect (ii). 
Helping realise and utilise strengths and weaknesses (iii). Building a sense of be- 
longing (iv). Mutual respect (v). Encouragement and care (vi). Role modelling. 
For peer feedback to be valuable, the qualified source needs to embrace some or 
all of these six strategies. 

Despite the benefits of peer feedback, there are also concerns about the qual- 
ity and accuracy of the feedback provided (Dearnley et al., 2013). Nevertheless, 
as peer reviews involve a non-expert evaluating another non-expert, the feed- 
back quality tends to be less satisfactory (Daalhuizen & Schoormans, 2018). 
Therefore, in some other experiential learning studies, peer feedback is jointly 
provided with teachers’ feedback to enhance its validity. For example, a group of 
first-year student teachers in Australia received both peer and mentor feedback 
on their teaching practices and reported increased awareness of the importance 
of participating in critical dialogues as part of their teaching (Daniel, Auhl, & 
Hastings, 2013). Assistance in the forms of prompt questions and discussion 
guidelines inspires students to review their peers’ work at greater depth, evaluate 
its quality more critically, undertake the review process in a more systematic 
manner, and consequently enhance their own learning (Frontczak, 2015). 


External Partners Feedback 


Industry and community partners are recognised as the most preferred and 
reliable feedback for holistic competency by future employers (Chan & Luk, 
under review). Equipped with relevant industrial knowledge and practical expe- 
rience, external partners are able to provide credible insights, suggestions, and 
advice that are difficult for academic teachers to replicate (Muir & Laxton, 2012; 
Voss & Blackburne, 2019). Given the importance of experiential learning, it is 
vital to ensure that there are structured mechanisms for external partners to pass 
on their feedback to students. 

Focusing on learning from experience, clients’ feedback enriches students’ 
perspectives and understandings of real-life work situations. Since clients are on 
the receiving end of students’ actions and decisions, their feedback is a reliable 
evaluation of students! performance. In a service-learning environment, the ser- 
vice recipients have ample opportunities for one-to-one encounters with student 
volunteers, and are able to offer more comprehensive and specific comments on 
students’ performance (Subramony, 2000). In some cases, clients’ and project 
partners’ comments are observation-based feedback (Harfitt & Chow, 2018; 
Isakovic, 2015), which can be used to inform academic instructors! evaluation of 
student achievement (Johnson & Brown, 2015). 

Some stakeholders may not be well versed in feedback skills. For example, 
in service-learning programmes, external agencies do not normally have proper 
mechanisms in place to provide constructive feedback to their volunteers (Sub- 
ramony, 2000). Hence, a partnership between on-site supervisors and academic 
instructors in providing continuous monitoring and feedback proves to be an ef- 
fective measure to ensure the successful implementation of experiential learning 
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programmes (Eyler, 2009). In fact, the use of multiple sources of feedback, or 
360-degree feedback, is well documented in experiential learning literature (e.g., 
Chilton, 2012; Miller et al., 2017; Powell & Bartlett, 2018), emphasising how 
different individuals such as academic teachers, peers, external mentors, and cli- 
ents can offer valuable insights into different aspects of a learner’s behaviour and 
performance (Subramony, 2000). 


Feedback from Multiple Sources 


A successful experiential learning programme should involve different quali- 
fied sources of feedback, as different sources bring out different perspectives 
from and on which students can critically reflect and improve. In a South 
Korean study, Shin et al. (2018) found students’ commitment and motivation 
to serve are affected by informative feedback from teachers and project di- 
rectors. In a work-based learning context reported by Dearnley et al. (2013), 
students also highly value the integration of feedback from peers, teachers and 
workplace stakeholders, and appreciate multiple sources of feedback. Dan- 
nefer and Prayson (2013) introduced a problem-based learning programme 
which further integrated self, peer and tutor feedback. Engaging students 
in their own experiential learning feedback could help them achieve better 
self-understanding and develop them into self-regulated learners (Dannefer & 
Prayson, 2013; Deeley, 2014). 


6.2.3 Types of Feedback Practices 


The literature has recorded many different types of feedback practices, including 
but not limited to written comments (Nicol, 2010), simple corrections (Mackey 
et al., 2007), and feedback talk integrated into daily teaching (Heron et al., 
2021). However, when attempting to ascertain whether certain practices can be 
considered as a type of ‘feedback’, we note that the boundary between assess- 
ment and feedback is not always clear (Chan & Luo, 2021b). In our recent paper, 
we examined whether university teachers (N = 248) recognised six types of com- 
mon pedagogical practices as feedback, and yielded mixed results. In particular, 
a lot of controversies surround whether ‘grades’ should be considered as a type 
of feedback or as an assessment result. In view of the convolution, Winstone and 
Boud (2020) emphasised the need to disentangle assessment and feedback, so 
as to “ensure that the legitimate purposes of both feedback and assessment are 
not compromised by inappropriate conflation of the two" (p. 1). It follows that 
feedback needs to be acknowledged as a meaningful act in itself, rather than as 
something that is used for grade justification or something that accompanies 
assessment. This substantiates my belief on outcomes-based approach as shown 
in Figure 3.2, in which feedback should be a separate component within the 
outcomes-based approach to the student learning model. Proper pedagogical 
design for feedback should be planned and not as an afterthought or conflated 
with assessment. 
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The six pedagogical practices discussed in our paper (Chan & Luo, 2021b) are 
listed below for readers’ reference. We argue that the choice of feedback type in 
experiential learning should depend on a number of factors, ranging from the 
purposes of the feedback, the timeliness of the feedback required, the wherea- 
bouts of the qualified source(s), the relationship between the student and the 
qualified source, to the action required by the student upon receiving the feed- 
back. (For the outcomes of the paper, please refer to the article directly. Note 
that as the paper does not focus on experiential learning, readers need to reflect 
on the suitability of these practices for experiential learning.) 


i Stamps/Digital badges 
ii Grades 
iii Simple corrections 
iv Rubrics 
v Comments to the whole class 
vi Generic exemplars 
vii Personal comments 


Experiential learning often occurs in locations outside the normal classroom and 
outside normal working hours. To overcome these challenges, and the challenge 
of providing prompt and constructive feedback, the advance in technology has 
come to our aid in enhancing feedback in the experiential learning scope. To 
name a few, in a service learning project reported by Deeley (2018), multiple 
technology-enhanced feedback provision channels were adopted, such as an on- 
line personal webpage space, video recordings, and audio-visual screencasting 
software. An earlier service learning study by Deeley (2014) also incorporated 
paper-based, PC-based, and mobile-based feedback. Deeley (2018) argued that 
the use of technology has allowed more detailed and speedy feedback, instead of 
relying merely on written comments; and the rich modalities (audio and video) 
have created more trust and deepened the bond between students and teachers 
by clearly communicating with them the tone, the expression and nuances in 
the given feedback. By doing so, the students can be expected to respond more 
actively to teachers! feedback and improve their learning. In Chapter 9, I will 
provide a case that demonstrates how technology can enhance experiential learn- 
ing assessment and feedback, and alleviate teachers’ workload. 


6.2.4 Feedback Literacy 


Similar to assessment and reflection, student and teacher feedback literacy are 
thorny concerns for useful feedback provision. Apart from enhancing interactive 
dialogues between qualified sources and the students (Ajjawi & Boud, 2017), 
student feedback literacy, i.e. “the understandings, capacities and dispositions 
needed to make sense of information and use it to enhance work or learning 
strategies” (Carless & Boud, 2018, p. 1315), is also necessary to achieve desirable 
feedback impact. Parikh et al. (2001)’s survey of 103 Canadian undergraduates 
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showed that many of the students did not find (or realise) the feedback received 
for their problem-based learning beneficial, and that is a clear sign showing us 
that more work needs to go into this area. 


Student Feedback Literacy 


In Carless and Boud’s (2018) conceptual paper on student feedback literacy, there 
are four inter-related components for student feedback literacy, namely, appreciating 
feedback, making judgements, managing affect and taking action. Students who 
are literate in feedback appreciate the usefulness of feedback, actively seek external 
evaluation, possess the ability to critically evaluate their own work and others, and 
can manage the socio-emotions that arise from feedback and feedforward to their 
future tasks among other learning-oriented perceptions and behaviours (Carless, 
2019; Carless & Boud, 2018; Han & Xu, 2020; Molloy et al., 2020). 


Teacher Feedback Literacy 


Teacher feedback literacy is defined by Carless and Winstone (2020) as “expertise 
and dispositions to design feedback in ways which enable student engagement 
and uptake”. In their conceptual paper (Carless & Winstone, 2020), they intro- 
duce a framework for teacher feedback literacy. The framework comprises three 
dimensions: design, relational, and pragmatic. The design dimension is related 
to the ability to design feedback tasks, promote peer feedback and evaluation 
judgement, and use appropriate feedback tools such as technologies, guidance, 
rubrics and exemplars. The relational dimension is teachers’ capacities to clarify 
the feedback purposes and processes, and demonstrate sensitivity and commit- 
ment towards supporting students and their learning. The pragmatic dimension, 
as the name indicates, is related to the actions that arise in the complexity of 
feedback. It “addresses how teachers manage the compromises inherent in dis- 
ciplinary and institutional feedback practices” (Carless & Winstone, 2020, p. 1), 
which includes handling tensions, balancing the teacher-provided and student- 
generated feedback, and managing compromises in workload and stakeholders’ 
satisfaction. Carless and Winstone (2020) believe that teacher and student feed- 
back literacy should be developed in tandem. Boud and Dawson (2021) com- 
plement this view, suggesting that “students are dependent on teachers having 
a good appreciation of feedback and how it can be deployed successfully” (p. 2). 
They created a teacher feedback literacy competency framework that presents the 
competencies required of university teachers at different levels of responsibility, 
from course design to providing student feedback (Figure 6.1). 

Feedback literacy, however, is not a natural ability. Students may notice a re- 
peated pattern in the comments they receive, but without proper guidance, they 
may not be able to act on these comments or apply them in their future work, 
thus failing to close the feedback loop (Quinton & Smallbone, 2010). Oppor- 
tunities for the development of such a capability need to be built into the de- 
sign of an experiential-based programme to make the most out of its learning 


200 Feedback in Experiential Learning 


ability to design feedback tasks, 
` promote peer feedback and 
Design evaluation judgement and using 
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Prag generated feedback, managing 
compromises in workload and 
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Figure 6.1 Teacher and student feedback literacy frameworks. 
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encounters (Moeller et al., 2019), specifically by allocating in-class time for re- 
flection after feedback is given (Guthrie & Jones, 2012; Quinton & Smallbone, 
2010). Guidance to students on the utility of feedback, in particular approaches 
for self-reflection and analysis of actions in relation to suggestions and con- 
structive comments, is held as a key feature of a meaningful feedback process 
(Runnacles et al., 2013). Structured feedback which contains explicitly stated 
components for observation and reflection can also be used to assist students 
to attain a deeper level of self-evaluation (Lavanya et al., 2016) and as a result, 
enhance their feedback literacy. The ability to understand and utilise feedback 
empowers students as autonomous learners to take responsibility for their own 
learning (García-Jiménez et al., 2015), which is a desired outcome and central 
focus of experiential learning (Perrin, 2014). The above suggestions point to an 
urgent need to improve teacher feedback literacy in experiential learning, which 
in turn enhances student feedback literacy. 


6.3 The Role of Feedback in Building Evaluative 
Judgement: Opportunities in Experiential Learning 


Another important focus in feedback research is how feedback can be implemented 
to support students’ evaluative judgement development in experiential learning. 
Evaluative judgement is defined as “the capability to make decisions about the qual- 
ity of work of oneself and others” (Tai et al., 2018, p. 467), the importance of which 
has been well-recorded in higher education. However, in recent years, there has been 
revived attention to researching evaluative judgement through a pedagogical lens, 
which has inspired a lot of new thoughts on the curriculum, such as how feedback 
may have the potential to develop student evaluative judgement but has yet been 
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effectively implemented for this purpose (e.g., Bearman et al., 2021; Boud et al., 
2018; Ibarra-Saiz et al., 2020). With a focus on developing evaluative judgement, 
feedback is expected to go beyond students’ work per se, and assist students in cal- 
ibrating their judgements of work by critically engaging with quality and standards 
(Tai et al., 2018). 

Evaluative judgement is not a particularly useful concept in judging factual 
knowledge that has rigid boundaries between right and wrong, but valuable 
in messy and authentic contexts where quality is difficult to describe and often 
subjective such as in experiential learning. Readers can refer to the work of Tai 
et al. (2016) and Bearman et al. (2021) in medical training settings, as well as 
Gladovic et al. (2021) in construction work, for some insights on how feedback 
can support students in developing evaluative judgement in experiential learn- 
ing. Meanwhile, as we know, holistic competencies are often central in experien- 
tial learning. Readers interested in understanding evaluative judgement of these 
competencies can refer to some of the work led by my postgraduate student, Ms. 
Jiahui Luo (Jess). An expanded framework on evaluative judgement of holistic 
competencies has been proposed based on a 3-round Delphi study with 14 in- 
ternational experts, with plenty of good insights regarding the role of feedback 
in shaping one’s evaluative judgement (Luo & Chan, 2022a). For those who 
are looking for a more “handy” guide regarding this topic, another paper of us 
(Luo & Chan, under review 2022b) may be helpful as it discusses assessment and 
feedback design from a teacher-oriented perspective. 


Conclusions 


Empirical studies that focus specifically on the impact of feedback in experiential 
learning contexts are relatively rare, and when the use of feedback is reported, 
the findings appear to be inconsistent. In some studies, feedback was shown 
to enhance learning outcomes (e.g., Cadotte & MacGuire, 2013; Konak et al., 
2014; Lavanya et al., 2016), whereas in others the findings were less conclusive 
(e.g., Alizadeh et al., 2017). Commenting on similar observations in their writ- 
ing, Steelman et al. (2004) conclude that the mechanisms of feedback are poorly 
understood. Hence, the use of feedback in experiential learning, particularly in 
terms of how post-secondary students receive and utilise feedback to improve 
learning, requires further research. 

This chapter synthesises important components of feedback mechanisms and re- 
lated issues from relevant literature on experiential learning in higher education. The 
discussion shows that the four components—quality of feedback input, qualified 
sources of feedback, types of feedback practices, and feedback literacy—are key fac- 
tors that influence student uptake of feedback and subsequently the utilisation of 
feedback for work, learning and life enhancement. In experiential learning, feedback 
also has immense potential to assist students in developing their evaluative judge- 
ment. Hence, the components and their sub-components provide a framework for 
understanding what an empowering feedback mechanism entails and how it can be 
effectively embedded in experiential pedagogy in higher education. 
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Questions to Ponder 


e Are you qualified to provide feedback? 

e What kind of feedback practices have you been using? Do they work? 

° Do you know the differences between critiquing and criticising? 

° If you were your students, what kind of feedback will you be asking 
for in experiential learning? 

e How can you provide students meaningful feedback in experiential 
learning that you know they will understand and use? 

e How do you build relationships with your students in order for them 
to feel comfortable with your feedback? 

e How do you help or train other stakeholders in the programme to 
provide constructive feedback in experiential learning? 

e How does evaluative judgement begin? 


Personal Reflection 


A few years ago, I wrote an article in the Times Higher Education pro- 
posing that every academic staff should have experience in industry, and 
suggested that maybe every few years academics should take leave to take 
on an industrial internship to boost their holistic competencies and indus- 
trial network. This will help teachers to become more qualified sources of 
feedback for experiential learning. A very daring controversial idea, what 
do you think? 

Read it here. 

http://tlerg.cetl.hku.hk/wp-content/uploads/2018/ 
02/Work-experience-should-be-a-job-requirement-for- 


academics-. -TH E-Opinion.pdf 
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7 Ethics in Assessing 
Experiential Learning 


The exact meaning of ethics is ineffable, but as an educator, we should at least 
know how to role model with integrity, honesty, fairness and responsibility. We 
should uphold the code of ethics to protect the rights of the students even when 


we design the assessment. 
— Chan, CKY 


Introduction 


Experiential learning is a powerful pedagogical tool. Apart from its active learn- 
ing engagement power, it inherently allows students to engage on more than just 
a cognitive level, but also on personal, emotional, and behavioural levels. And 
it is exactly because of these richer yet uncertain engagements that teachers and 
designers of experiential learning activities need to pay special attention to the 
ethical issues that arise from experiential learning assessment. 

Although my intention for this book is to focus on the assessment of experi- 
ential learning, it is difficult to separate the ethics of learning activities from as- 
sessment. Often, experiential learning teachers and designers overlook the basic 
ethical dilemmas that are inherent in teaching. Some of these ethical dilemmas 
are innate in experiential learning within its “experientialism and experimenta- 
tion” (Ward & Gomolka, 1989), and these dilemmas affect both the pedagogi- 
cal approach and the assessment. 

Most ethical studies in the context of experiential learning are associated with 
how experiential learning can enhance student’s awareness and understanding of 
ethics. A study by Teixeira-Poit et al. (2011) presents a useful three-step exercise 
to teach research ethics via experiential learning strategies. The three steps in- 
clude, firstly, providing students with an overview of selected ethical guidelines, 
after which they are presented with a fictional case study with an example of a 
researcher violating ethical guidelines, and finally, an interactive exercise, com- 
prised of a role-playing activity, is facilitated. In another study by Sims (2002), 
he suggested educators to pay more attention in the design of systematic and 
analytical debriefings in the experiential learning exercises for business ethics 
courses. Such debriefings can “insure an integration of the experiences with 
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concepts and applications to outside situations so that appropriate generaliza- 
tions can be made” (Hunsaker, 1978, cited in Sims, 2002, pp. 181-182). These 
are useful techniques for encouraging students to pay more attention to ethics 
in their field of study or in their future profession. However, those techniques 
are not addressing ethical issues inherent in experiential learning that students, 
teachers or other stakeholders in experiential learning may face. Under neolib- 
eralist education, there is great pressure from different stakeholders and their 
demands, such as the emphasis on authenticity and relevancy to the real world, 
transferring more responsibilities to students for equal partnerships, and devel- 
oping lifelong learning in ill-defined problems, and experiential learning has 
become the solution to many of these demands. Under such pressure, we may 
have grown complacent in acknowledging the ethical issues that arise from the 
genuineness of experiential learning. 

In this chapter, I present some of the ethical dilemmas that teachers and de- 
signers face in experiential learning. Ethics is not simply about right or wrong, 
and it depends on many factors including one’s culture, religion, and beliefs. 
Often, the standards are based on a society’s accepted beliefs and principles (the 
norm) under which people live and work in harmony with one another. It is not 
always easy to identify these ethical dilemmas; thus, case studies will be provided 
to better inform teachers about them in the context of experiential learning. Fi- 
nally, suggestions will be included to help teachers and designers of experiential 
learning to overcome some of these concerns. 


7.1 Ethics is ... 


Ethics are beliefs and principles that an individual possesses to guide one- 
self to evaluate what is right and wrong in an environment. In short, it is 
the imaginary moral line that one would have internally to uphold their 
actions. 

Chan, C.K.Y. (2021) 


These values are usually an extension of one’s personal values such as honesty, re- 
sponsibility, and integrity, and are unique to each person based on one’s family, 
culture, environment, religion, and ethnicity (Poorchangizi et al., 2017). 


7.2 Ethical Dilemmas in Experiential Learning 


Teacher’s Code of Conduct 


Most university teachers are unaware of the code of ethics as a teacher. While 
professional development workshops on research ethics are often compulsory for 
university teachers, teachers’ code of conduct are rarely covered in any workshop. 
Universities also do not require teachers to sign a code of conduct. In this sec- 
tion, I will discuss the ethical dilemmas often found in experiential learning for 
teachers to be more aware of their code of conduct as a teacher. 
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Forced and Optional Participation 


Experiential learning provides student with meaningful learning opportunities. 
However, some students may opt to avoid these activities as they may not wish to 
put effort into developing non-disciplinary or non-academic-related skills, especially 
when teachers may use unconventional assessment methods which could make it 
harder for them to achieve high grades. Some experiential learning courses are com- 
pulsory, and students who are forced to participate may not share the same enthu- 
siasm and emotion as those who hope for a meaningful experience. Even for those 
who enrol voluntarily, we may not have evaluated the suitability of the assigned 
activity for each individual participant. Say a student grew up in an orphanage and 
was asked to share their success story with others from a similar background. As 
their teacher, have we considered the student’s feelings? Yet, teachers’ decisions of 
who should be involved in and who should be exempted from the activity can create 
an ethical dilemma. Kate Trimble, a senior associate dean and director of the Office 
of Experiential Learning at MIT, runs an experiential ethics course, and she agrees 
(Durant, 2020) that “what is not going to work is for students to feel like ethics 
is being pushed on them” and ethics is like “a muscle that you develop over time, 
with repeated exposure, in different contexts, and by talking to people who have 
different perspectives and experiences than you do.” 


Teacher’s Biased Experience, Perception and Expectation 


Every teacher has their own perception and expectation of how they teach. Their 
experience also determines what they anticipate their students to achieve. While 
such subjective perspectives can be minimised within traditional classrooms due 
to common standards, these biased expectations and perceptions can create far 
greater risks and issues for students in experiential learning. For example, as part 
of the assessment in an entrepreneurship course, teacher arranges students to pur- 
sue an actual entrepreneurship competition organised nationally by business and 
venture capital organisations. It requires each team to develop a business proposal, 
a presentation and the prototype of the product based on a specified theme. The 
teacher provides a template for the business proposal, insisting that all students 
must use the same template for ease of assessing, and students who do not follow 
the guidelines will be disbarred from entering the competition as it requires a 
teacher’s nomination. Students may question the accuracy of the business template 
as compared to the expectations of the judges who are genuinely from the business 
sectors, and it is unethical for students to be punished and prevented from further 
participating in a meaningful learning experience due to subjective preferences. 


Honesty in Assessment 


Assessment in experiential learning can undermine the meaningfulness of the 
activity. Even if an assessment is designed to be authentic, it may not necessarily 
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elicit truthful responses. For example, for compulsory reflective assessment, stu- 
dents may fabricate their reflection to hide their true feelings, as they may feel 
pressurised into producing an assessment product that is acceptable, rather than 
truthful (Chan & Lee, 2021; Cotton, 2001; Newcomb et al., 2018). 


Student Evaluation and Teacher’s Ethical Responsibilities 


Experiential learning can produce great student outcomes to showcase as evi- 
dence of student learning or used for research purposes. However, teachers need 
to be reminded that they have an ethical responsibility toward the students. 
Teachers need to get consent from the students before revealing their work to 
external audiences. 


Emotion Associated with Experiential Learning Assessment 


Strong personal emotions often arise from experiential learning experience. 
These sentiments and reactions can be so intense that they change a student’s 
self-perception. Teachers sometimes neglect these important emotions (which 
could also be intended outcomes) and set inappropriate assessments for the ex- 
periential learning exercise. Reflective assessment is one of the most widely used 
assessment approaches in experiential learning, which can be effective, but may 
sometimes be too invasive and “demand confession and self-surveillance as evi- 
dence of progress and learning” (Ross, 2011, p. 115). 

In a nurse education study by McCarthy et al. (2018), they described a study 
on students’ self-reported reflective assessment in which students reported al- 
leged malpractice with the clinical staff. This has caused conflicts between clin- 
ical and academic staff. Thus, implications need to draw in place to avoid any 
conflicts and reprimands. 


Student’s Self-Disclosure 


One of the main ethical issues arising from reflective assessment in experiential 
learning is the vulnerability and disclosure of personal information that is ex- 
pected from students. As previously mentioned, literature (Chan & Lee, 2021) 
has repeatedly found that students feel wary or adverse towards the prospect of 
their reflections being read and assessed by their teacher, peers, or supervisors. 
There are risks associated with reflections and it can be harmful to students 
when they reveal or discuss their weaknesses (Hickson, 2011), or when a neg- 
ative focus is taken, which, in turn, damages their self-confidence (Knowles 
et al., 2001). 

Teachers also face ethical dilemma when students disclose abuse, malpractice 
and negligence in their reflection. Their position may cause them to struggle 
between confidentiality and ethics as well as legal and societal responsibilities 
(Cleary et al., 2013). 
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Accessibility of Reflections Made in Public Spaces 


Public spaces such as virtual social network spaces are also a concern for experiential 
learning assessment. Both Ross (2011) and Smith and Trede (2013), pointed out 
that online journaling has become increasingly influenced by the trends of blogging, 
which is associated with a degree of self-promotion and social media presence, and 
is not necessarily reflective. The emotional challenge and authenticity of reflections 
made on the online platforms, the extent to which the reflections posted and shared 
on such platforms may affect genuine reflectiveness, and of course, the privacy and 
safety of students are all part of this concern. It is also difficult to know if the stu- 
dents are really reflecting or if they are simply seeking attention, knowing all their 
friends will be reading their entries? This brings back the issue on honesty men- 
tioned above. Teachers should not overlook the need for social acceptance and the 
effect of peer pressure when they design the assessment. 


Complex Relationship 


The relationship between students, teachers and other relevant assessors in experi- 
ential learning is a complex one. There can be passion, excitement, sorrow, friend- 
ship, sympathy, empathy and many other emotions developed between teachers, 
supervisors, mentors and students. These bonds become intensely complex when 
assessment (a grade) is required. For the assessor, questions in mind might include: 


e AmI being fair to give them a good grade or is it because they are my friend? 
e Are they being friendly and helpful because they know that I am the 
assessor? 


The power dynamics between all the different stakeholders may also cause chal- 
lenges. Students may have internal struggles such as: 


e Idon’t agree with my work supervisor (in an internship). I don't know them 
at all, they are never around, I have only met them once, but they are going 
to do an appraisal for my internship, should I just go along with their ideas 
in case they jeopardise my grade? 

° Personal reflections are private, why should I share them? But I have a good 
relationship with my mentor, maybe it is ok to share with them? 


7.3 Case Studies in Experiential Learning 


In Chapter 1, we stated that in its simplest form, experiential learning is learn- 
ing by doing, and for ethics in experiential learning, understanding must also 
be generated through experience. However, it helps to present some case stud- 
ies that allow you, the readers, to ponder through and realise that the actual 
standards for professional values and ethics in experiential learning may differ 
greatly between you and me, depending on our culture, religion, beliefs, disci- 
pline backgrounds, political views and many more. I am not presenting any right 
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or wrong answers here, but I hope that by presenting these case studies, you can 
ponder through them (or perhaps discuss with your students and colleagues) and 
make your own judgement. These case studies will be useful in professional de- 
velopment workshops for academic developers to help teachers to become more 
assessment literate in experiential learning. 


7.3.1 Case Study 1 — Internship 


For an internship course, students are asked to put together simple daily notes (as ev- 
idence) and use the notes to prepare a reflective essay of 1,000 words to be handed in 
as the final product for the course. This will be followed by a feedback interview with 
both the internship supervisor and the teacher. The reflective essay should focus on 
the outcomes that the students achieved, the challenges they overcame, the type of 
career they want to pursue and the kind of person they want to become. 

A female student who completed an internship at an international consultancy 
company submitted her reflective essay. She has written about her insights in a 
professional environment and how she applied her knowledge from her courses 
in her work. She was proud that she built up resilience during her time at the 
company as she had to overcome a major hurdle that has seriously troubled her. 
She did not explicitly mention the major hurdle, but she implied that she was 
sexually harassed by a senior employee in the company. In the reflective essay, 
she did not go into details about the actual incident; instead, she reflected on her 
own behaviours, how she may have done differently next time, and wrote very 
professionally on how she sees herself in the future if such an opportunity in one 
of these international companies arises. She emphasised that she now realises 
that the world she lives in is a dog eats dog world. 

The course coordinator who received the reflective essay did not teach the stu- 
dent previously, there has never been any previous encounter with this student. 
No sexual harassment report has been filed. The teacher does not know how to 
proceed. If you were the course coordinator, will you do one of the below options? 


1 The teacher can treat the reflective essay as a fiction, ignore the actual con- 
tent and mark it based on outcomes developed. Then the teacher can invite 
the student for an interview (without the internship supervisor) to avoid any 
discomfort. 

2 The teacher can invite the female student for a private conversation and dis- 
cuss with her on the situation. If the teacher is a male, a female staff should 
be present. 

3 The teacher can inform the relevant university authority offices such as stu- 
dent affairs, counselling office and faculty dean to discuss the seriousness of 
the situation. This approach may cause a lot of discomfort to many stake- 
holders and therefore it needs to be handled with sensitivity. 


e The female student will need to be interviewed (by relevant authorities) to in- 
vestigate if the actual incident has occurred. The female student may not wish 
to disclose the information for numerous reasons. She may also have (or not) 
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fabricated the story for a more interesting reflective essay. Parents may need to 
be involved. The student may sever her opportunity to receive an offer from that 
and other international companies indefinitely. 

e The internship supervisor may also be invited for an interview (by relevant 
authorities), and he may be penalised by his company for not knowing about 
the incident and not conducting his job in a professional manner. 

e The relationship between the university and the company will be damaged. 


What are the Ethical Dilemmas in this Case? 


° Is the teacher obliged to report this? 

e Will there be disclosures that the student will later regret sharing? 

e Will this sabotage the student’s career opportunity in life? 

e Would this become problematic for the teacher? For example, their relation- 
ship with the company or faculty heads? 

e Would the university be liable if harm comes to the student as a result of 
breaches? 

° Isa reflective essay an appropriate assessment for an internship? 

e How important are the truthfulness and the facts of the reflective essay? 

e Is it ethical to expose the student to the issues above under an imbalanced 
power relationship? 


7.3.2 Case Study 2 — Non-Governmental Organization (NGO) 
Community Service Project 


To obtain a social science degree at a university, students must pursue a five- 
week community service project as it is a compulsory part of their graduation 
requirements. Students may opt to travel overseas or stay locally and work with 
a congressman, in homeless and health organisations, legal areas, women’s ser- 
vices, start-up organisations, refugee or immigrant services. The course coor- 
dinator cannot assess students’ everyday work or projects as the teacher cannot 
oversee all the projects. Thus, the community supervisor will assist, and a post- 
graduate student will act as a mentor. As part of the assessment, students are 
asked to use videovoice (similar to video diary) to document and share the lived 
experiences of their clients to raise the clients’ consciousness and to influence 
decision-makers to affect community-level change (Catalani et al., 2012; Chavez 
et al., 2004). 

A male student opted to travel to Indonesia and work with the local Chamber 
of Commerce. As part of the project, the student must live with a host family 
in Indonesia in order to immerse in the culture. Every host family receives a fee 
from the university. The student was treated well during his stay, but he was 
petrified with the family’s own living conditions. He also witnessed some brutal 
behaviours towards the females by the male parties in the household. He also 
realised that bribery with the local government including the local Chamber of 
Commerce forms part of the business tradition. The student was able to video 
record and interview many instances with the family and the local businesses. 
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Some of the locals described the corruption process in details. The student has 
received informed consents from those in the videos, he has also agreed not to 
disclose their true identifications and redact any sensitive information. 

The student received top marks for his project and the course coordinator 
wanted to use the case as a champion piece to showcase community service pro- 
jects in the social sciences discipline at the university. 


What are the Ethical Dilemmas in this Case? 


e Do the individuals truly understand what an informed consent is? 

e Would the true identifications of the individuals in the video cause harm to 
them? 

e What would be the consequences if the true identifications or information 
about the corruption is publicly promoted? 

e Would there be any negative emotions from the student? 

e Is the teacher setting a good role model for the student by using this project 
as a showcase for his own benefit? 

° Is there an emphasis that the mark for the project is more important than all? 

° Isit ethical to expose the individuals to the above-mentioned risks under 
the circumstances? 


7.3.3 Case Study 3 — Legal Clinics 


Many universities have launched legal clinics to provide pro-bono legal advice for the 
public. Services are provided by law students which presents them the opportunity to 
get hands-on with real legal cases and clients. The legal clinics may provide legal con- 
sultation services in areas such as refugee and human trafficking, migration, disabil- 
ity rights, equality rights, innovation and entrepreneurship, family law, inheritance, 
criminal actions, and property and employment law. Students will be randomly as- 
signed to different cases to maximum their exposure to different areas of law. 

Students will be assessed via observation by the supervising staff of the legal 
clinic and the clients on their professional attitudes and competence when they 
handle clients? cases. Students are also required to present a five-minute case 
study on what they have encountered at the legal clinic to share with the class, 
which will be followed by a Q & A session. Both teachers and students will act 
as assessors for the case study presentation. 

A female student of Indian ethnicity was assigned to a human trafficking case 
in which the female client, the mother of the victim was accused as the primary 
familial trafficker in this case. At the Q & A session, it was found out that the 
female student came from a family with strong traditional values and where 
arranged or forced marriage is not uncommon. The Indian female student is 
against such traditional values. 


What are the Ethical Dilemmas in this Case? 


° Is the client the best assessor on professional attitudes and competence in 
this case? 
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Is presentation an appropriate assessment approach in this case? 

Is Q &A session necessary? 

Will the female student be harassed by other students who simply do not 
agree with the female student on how the case was handled? 

How can the teacher ensure that the class handles personal information 
maturely? 


As mentioned, I invite you to work with your students and colleagues to deduce 
your own standards in ethics. In the next section, I will provide some sugges- 


tions on how to go about dealing with ethical issues in experiential learning 
assessment. 


7.4 Suggestions to Overcome Ethical Issues in Assessing 


Experiential Learning 


Trusted Environment 


Propose “a learning environment and a portfolio that values a reflective 
dialogue with a trusted person in an open and safe way” (Driessen, 2017, 
p. 225) would help solve some ethical issues in reflection. For example, hav- 
ing trained senior students as mentors can help with a trusted environment 
(Chan & Luo, 2020) 

Organise regular meetings with students either individually or in small 
groups to build trust and safeguard student wellbeing (Stewart & Richard- 
son, 2000) 

Call for more structured guidance and support network for assessment de- 
sign in experiential learning with clearer rationale (Sutton et al., 2007) 


Professional Development for Teachers and Students 


Code of ethics and professional development workshops on ethics for all 
university teachers teaching at the university 

Use cases to explore ethical questions in journal writing and provides prin- 
ciples for using it in educational settings (English, 2001); 

Explain the benefits of experiential learning to students and how the as- 
sessment may help them reframe their experiences. It is essential to pre- 
pare students for the practical and emotional difficulties that they might 
experience 

Engage in an ethical dialogue about the use of assessment practice 


Assessing Experiential Learning 


Enhance students’, teachers’ and assessors’ assessment literacy in experien- 
tial learning 
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Use formative assessment in reflective journals (Creme, 2005). Reflective assess- 
ment can be a process instead of a product; use simple pass/fail grade instead of 
actual percentages if it is to be assessed (Stuwart & Richardson, 2000) 

Invite students to reflect on fabricated professional scenarios, so they do not 
have to reveal their actual experiences (Hargreaves, 2004) 

Ensure teachers design assessments with ethics in mind. The ethical dilem- 
mas in Section 7.2 can be used to guide you through 

Let students know what, who, how, and why are you assessing 

Depend on the activity and the relationship, have teachers or the public 
NOT directly involved in the course may sometimes be more suitable to 
conduct the grading (Sutton et al., 2007) 

The locus for reflection is not “in” the individual (decontextualised), but 
“in” the total event (Ross, 2014). For example, students should not be 
graded based on their individual experiences or feelings, but on what they 
have developed, improved and become 

Enhance assessors’ feedback skills 


Recommended Guidelines 


Devise a policy or guidelines for students and colleagues on how best to 
proceed when ethical concerns arise in student assignments; attend to all 
rights of concerns that may arise 

The ethical policy or guidelines should be presented to the students at the 
beginning, providing clear direction on what is expected and required, and 
on the importance of adhering to the ethical principles, particularly on 
confidentiality and anonymity. If personal disclosure is expected to share 
among the class, students in the class need to adhere to the rules completely 
Develop a policy for disclosure 

Develop a referral network with university support services, making them 
easily available for students 

The policy or guidelines need to provide clear procedures on how teach- 
ers should address ethical ambiguity (McCarthy et al., 2018), given many 
ethical dilemmas are unclear, and often not necessarily black or white. For 
example, is it ethically wrong to ask a student who has experienced hardship 
in their life to share their stories, would that be inconsiderate, or is it a test 
of resilience and maybe even a triumphant moment for the student? 


Conclusions 


This chapter provides a review of the empirical and conceptual literature on 
ethical issues concerning experiential learning in higher education. To help ed- 
ucators to be more aware of ethical issues when designing experiential learning 
assessments, case studies were introduced. Some of the dilemmas presented in 
the case studies cannot be easily eliminated, but experience, good faith and prac- 
tice on our part can help to avoid some of them (Figure 7.1). 
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Figure 7.1 Imitation of a teacher of the lady justice (Drawing by Aisling Colloton). 


Question to Ponder 
For this chapter, I only have one question for you to ponder. 


° Always raise the question to yourself, if you were in their shoes, is this 
acceptable to you? 


Personal Reflection 


My quote at the beginning demonstrates a very important point, how we 
as educators act as role models. Students are accustomed to follow, to de- 
pend on and to be influenced by teachers. Our roles are not just about 
teaching, we should be modelling for our next generation the act of integ- 
rity, honesty, fairness, responsibility and principles. 

You can choose to be the educator shaping our society to be a better world. 
I really like Aisling Colloton’s imitation of a teacher of the Lady Justice. 
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8 Assessment Cases around 
the World 


According to Oscar Wilde — ‘Experience is the hardest kind of teacher. It gives you 
the test first and the lesson afterward.’ For teachers, we may not always have that 


luxury to test all our courses, Thus, we borrow ideas from authentic cases, combine 
with our experience and reflect on NOT just our experience but others. 
— Chan, CKY 


Introduction 


In this chapter, I put together many excellent 
cases of experiential learning courses and activi- 
ties from universities around the world. These 
cases focus on the learning outcomes and assess- 
ment approaches used. Each case consists of a brief 
description, the desired learning outcomes, the 
teaching and learning activities and the assessment 
approaches (and most of the time, the weighting 
of each approach, the aligned outcomes, the tar- 
geted holistic competencies and sample rubrics are 
included). In addition, the teacher’s background 
and motivation, the challenges they encountered and the stories from students 
who participated are also put together. There are also a lot of videos and online 
references attached to each case. For ease of access, I have included our Assess- 
ment Resources weblink (https://ar.cetl.hku.hk) and a QR code which are con- 
nected to the video of these cases. 

These selected cases are from Hong Kong, Singapore, Australia, New Zealand, 
Canada, the United Kingdom and the United States with some faculty-based 
and some cross-disciplined. As each experiential learning course and activity is 
unique and often with individual characteristics, hopefully, teachers can borrow 
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ideas from the different cases and build their own unique assessment. Due to 
space constraints, I could only display 12 case studies here, but you can find 
more information on the website using the QR code above. 


8.1 Case 1: “Transformative Business Immersion in 
Developing Economies”, Community Services, Business 
School, University of Hong Kong, Hong Kong 


Offered in summer term, the course “Transformative Business Immer- 
sion in Developing Economies” (BUSI, 2816) is a community-service trip 
blended with a business consultation practicum. This course provides stu- 
dents with a total immersion experience in an unfamiliar geographical, 
economical, and social environment. After a week of preparation at the 
University of Hong Kong (HKU), students travel to a pre-selected loca- 
tion in the Philippines where they consult with local micro businesses, 
transferring fundamental business principles to resolve specific issues that 
the company faces. Students live with local families in a homestay arrange- 
ment, so as to quickly understand the local culture, customs and business 
practices. 

After initial scoping trips conducted by the course developer, the trial 
run of this course was completed in 2015 when it was only open to 15 
students in the Business School. It has subsequently expanded to two sub- 
classes per summer (in June and in August), accommodating a total of 60 
undergraduate students from any disciplines of Year 2 or above. 


Distinctive Features: 


e Enabling in-situ experiences of the developing world 
e Innovating solutions to real-world problems for micro-businesses 
e A combination of cultural immersion and business practicum 


Designed Learning Outcomes (LO): 


I Perform a detailed analysis of the operations of a small business in an 

emerging economy; 

H Apply theoretical knowledge learned in past courses to transform it to 
fundamental business; 

III Analyse the effectiveness of past course consulting engagements 
through interviews and data gathering; 

IV Develop a deeper, more thoughtful understanding ofthe culture, econ- 
omy and environment through daily interaction with host families. 
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Coursework Teaching & Learning Activities: 


Pre-trip preparation: 


e ` Self-study (with a reading pack) [20 hours; Week 1] 
e Class discussion [40 hours; Week 1] 


Business trip in the Philippines: 


Week 2 — intensive information gathering & investigation 

Week 3 — creation ofa written report on the business principles and specific 
issues to focus on with proposed solutions 

Week 4 — plan execution with local business manager/owner 


e Interviews & meetings [30 hours; Week 2—4] 
e Analysis & report writing [40 hours; Week 2-3] 
Source: HKU (2017) BUSI2816 Course Outline. 


Assessment Approaches 

Name Learning Activities Weight Aligned Type Generic Skills 
LO 

Al Analysis of Students conduct 20% T. Formative Collaboration, 
previous interviews and data TT. Communication, 
consulting gathering to determine Critical thinking, 
engagements the impact and Study skills 

effectiveness of past 
course consulting 
engagements. 

A2 Business Students are required 40% I. Summative Collaboration, 
Analysis and to produce a detailed (88 Numeracy, 
Solutions report on the sector IV. Problem-solving, 
Report and business, and their Study skills 

ideas for solutions or 
improvements. 

A3 Video Students prepare a 20% Il. Summative Communication, 
Report photographic and/or IV. Creativity, IT 

video report of their 
time abroad. 

A4 Reflective Students actively 20% IV. Formative Communication, 
blogging contribute to a group Self-management 
(weekly) blog that will share 


their experiences with 

the homestay and with 
their selected business 

projects. 
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Assessment Type 


Both formative and summative assessments are employed. The formative as- 
sessments set out the initial state of practices and keep track of field progress 
in multiple dimensions. The summative ones summarize cultural and academic 
learning respectively. 


Main Assessment Areas 


Community Virtues: The course design enables cultural exchange and 
insights to life within a developing economy. Students’ interactions with 
host families and local communities are part of the assessment stand- 
ards, which can be evidenced with video reports and reflective blogging 
(A3 & A4). Students are expected to show their sense of empathy and fair 
reciprocity while being interactive and open-minded. The service-based 
learning hopefully contributes to the improvement of not only a single 
business, but also the economy of the town that the class visits. In the 
trial run of the course in 2015 and in June 2016, the group stayed in 
Jordan, Guimaras; in Aug 2016 and 2017, the group turned to Jagna, Bo- 
hol. These are two towns separated by different islands in the Philippines, 
about 300 kilometres apart. The batches of student visits thus contribute 
to different local communities with their business service learning. 
Self-management Skills: Adaptive self-management is vital as students 
have to adjust to an unfamiliar living situation “outside of their comfort 
zone” with few infrastructural supports and high demands of independ- 
ence. As reflected in video reports (A3), students are “pushed to be at 
their best.” They are expected to take initiatives in proposing new ideas 
and handling problems. How they deal with everyday situations abroad 
can be recorded and assessed in their reflective blogging (A4). Through 
living and working abroad, students have realized the importance of “self- 
analysis and introspection,” which are indispensable traits of personal and 
career development (Lefler, 2016). 

Problem-Solving Skills: Both problem identification (A1) and proposed 
solutions (A2) are assessed. Students need to deal with unforeseen prob- 
lems when they help the local businesses execute the proposed plan, such 
as situations of reluctant business owners, challenging home stays with 
compromised bathrooms and lack of air conditioning in constantly high 
temperatures, or simply the slow internet. When students get to identify 
the particular problems that local businesses are facing, some of them re- 
alize that it could have been a “make-or-break situation" for the business 
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owners (Lefler, 2016). The problem identification process is thus also an 
analytical understanding practice. 

Communication Skills: An essential requirement for providing mindful 
consultation to local businesses is to properly understand their needs and 
operational issues in their perspectives. In addition to expressing oneself 
effectively, the student needs to be a patient listener to understand the 
indigenous context. Interviews with local businesses and class meetings 
are arranged to investigate the situation, where communication is the key 
technique. Students have identified that “the technique of question asking 
and rephrasing” is important in communication (Lefler, 2016). Analysis 
reports (Al & A2) are compiled based on data/information gathered from 
the field through students’ proactive communication. Moreover, how the 
student interacts with local community and the host family is part of the 
grading standard. 

Collaboration Skills: Throughout the time abroad, students work in 
teams. Students collaborate with each other in field investigation to per- 
form business analysis, in writing up reports (A1&A2), and in producing 
a video report (A3) that summarizes work-life experiences abroad. In the 
expanded course, normally eight students work together in one group, 
as it is practical to learn how to steer up and manage group dynamics to 
maintain productivity. Students can itemize the assigned tasks and divide 
the work among team members to achieve optimal outcomes. 

Knowledge Application: During the course of consultation, students 
get to see all aspects of a business and how it functions. Fundamental 
business conundrums, such as volatile local operations with challenges 
resulting from macro-economic conditions, are presented to the students 
at a small enough scale, so they can manage to wrap their heads around 
the concept. The students are then able to apply their formal training 
and education to the real world, as well as to reflect on their theoretical 
understanding with practical experience borrowed from the owners and 
employees (A2). 

Reflection: The photographic/ video report (A3) is due two weeks after 
the students return to Hong Kong, allowing sufficient time for them to 
further reflect on their encounters. The weekly blog posts (A4) are re- 
quired to be reflective in nature, covering different areas of work/life 
abroad, from consulting engagement and homestay, to cultural learning. 
Students are given ample “time and space to deepen their understanding 
of their experience and how it fits into their worldview” (Lefler, 2015). 
The experiences in the developing world make the students reflect on the 
meaning of life and value of happiness. 
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Teacher’s Stories 


Beau Lefler 


Professional Engagements 


After obtaining a Juris Doctor in the United States, Beau practiced law 
before joining HKU. He now primarily teaches business law and com- 
pany law and advises students in the Law program of Bachelor of Business 
Administration. 

After soliciting funding from HKU's Teaching Development Grant and 
the Gallant Ho Experiential Learning Centre, Beau began running the 
course, this was since 2015. 

This highly immersive course brings together Beau’s goal of enabling 
students to understand real life and the developing world while providing 
them an opportunity to use their educational training to do something 
valuable that altruistically benefits others. During their time overseas, he 
also wants his students to develop a consciousness of different cultures and 
learn how to take new perspectives while having fun. As noted by Lefler, 
“Pm interested in helping the students understand real life; I want them to 
understand [the] developing world, cause we’re quite arrogant sometimes 
in the developed world” (May 2016, teacher interview). 


Motivation 


In recent few years, Beau has seen a growing number of students seek- 
ing opportunities to do something beneficial and worthwhile with their 
careers, but they are often unable to find the right outlets. Combined 
with Beau's interest in development economics — specifically about how 
countries can be helped to develop and to offer better living conditions, 
this status quo motivated Lefler to design and set up the Transformative 
Business Immersion in Developing Economies course. 

Keeping students’ costs to the minimum, Beau also seeks private cor- 
porate sponsorship to support the future operation of the course. To not 
close off the opportunity to students who are frustrated financially, suffi- 
cient funding needs to be procured. 


Challenges 


It is difficult and time-consuming in finding enough host families and 
local businesses within the same remote area to make course arrangements 
with. While the families can be reimbursed for the costs of hosting the 
students during their stay (e.g. electricity, food, mosquito nets and cots 
for the students), additional money exchange for these arrangements is 
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avoided as it might ultimately take away the true virtues of the homestay 
experience — to build meaningful relationships. Destinations where there 
is no need for translators are prioritized, where closer relations with host 
families and businesses can be built without additional language barriers. 


Students’ Side of Stories 


e “We tried to identify the problems that Z-Network (a local shop 
selling and servicing computer parts) faced, but before that we 
needed more information. Unlike in Hong Kong, nothing was 
available online, so we had to go to (government) offices, shops 
and schools, and asked questions. We collected names, contacts, 
addresses, and anything else we could find. After discussing with 
professor, and other teammates, we found that Z-Network needed 
a strategy to target new customers, and to maintain client relation- 
ships. We developed a marketing plan, after piloting a few different 
strategies. We also developed a customer database which Z-Network 
can utilize to maintain and initiate customer relationships.” (Video 
Project, June 2015) 

e “Throughout my stay, I felt like I was really a part of the host family, 
because they look after me like I am one of them. From night swim- 
ming to slaying a chicken, they have introduced me to things that I 
have never experienced before. Staying in Philippines was frustrating 
at first, the slow internet, compromised bathrooms, constantly high 
temperatures. But once I got over these superficial circumstances, I 
was able to view my life from an objective perspective that I could 
not gain in the busy life of Hong Kong.” (Team Blue, Video Project, 
August 2016) 

e “We learnt to look at small businesses at a different perspective, and 
to understand their challenges before providing feasible recommenda- 
tions. We helped businesses with simple accounting records, and the 
designing of logos and banners, in hope to improve their profitability 
and the economy of the town. This course has not only provided a 
platform for us to practice our business knowledge, it also allows us 
to go beyond just building client relationships to establishing valuable 
friendships in the process. This is indeed an unforgettable learning 
experience." (Group 2, Video Project, June 2017) 

e° “Its been a crazy ride, each of us with varied experiences. We all had 
problems like reluctant business owners, or challenging home stays, but 
T'd say we did a pretty good job overcoming them. And it's these chal- 
lenges that have enriched us.” (Group 1, Video Project, June 2017) 

e “Entrepreneurship is no joke, most of these businesses (referring to 
the ones in the business center of Jagna) don't even know the meaning 
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of this word but they live it in its truest form — groceries, snack shops, 
tailors and other vendors. We were able to immerse ourselves into 
the micro economies of this town, be it fixing accounting systems, 
designing logos or internet marketing, we tried to do our bit to make 
their lives just a little easier.” (Group 1, Video Project, June 2017) 


Featured Videos 


e Group video of students (Team 1) from Aug 2017 class https://youtu.be/ 


zpuqr5qydtc 

e Group video of students (Team 2) from Aug 2017 class https://youtu. 
be/4DjYfAQ7vJg 

e Group video of students (Team 2) from Jun 2017 class https://youtu.be/ 
Kiss NhWgdoo 


e Group video of students (Team Red) from Aug 2016 class https://youtu. 
be/PMrHb8yk5-E 

e Group video of students (Team 1) from Jun 2015 class https://youtu.be/ 
SZzRK9ycZoc 

e Group video of students (Team 2) from Jun 2015 class https://youtu.be/ 
un4Uu2sPttM 
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8.2 Case 2: “Build Your Own Airship Experiential 
Learning”, School of Engineering, Hong Kong 
University of Science and Technology, Hong Kong 


An “airship project” has been devised as an instrument of experiential 
learning by Hong Kong University of Science and Technology (HKUS- 
T)’s School of Engineering. In a course with the airship project, students 
work in teams to design and build their own airships that can be remotely 
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controlled by Android smartphones, using helium balloons, microcon- 
trollers, circuit boards, motors and propellers. These airships then en- 
ter an internal competition to perform certain tasks (e.g. weight lifting, 
transporting). Three different versions of the airship project have been 
developed: 


l A First-Year Cornerstone Engineering Design Project Course 
(ENGG1100) as an elective introductory course for Year 1 engineer- 
ing students to explore various areas of engineering (mechanics, pro- 
gramming and electronics) while developing essential teamwork skills 
for university studies and future careers 

2 An Engineering Team Design Experience: Airship (ENGGI200) 
course as a common core for all undergraduate students outside the 
engineering discipline to enhance their creativity and teamwork 

3 A series of community engagement activities with students from local 
secondary schools to experience and prepare for university life 


The class size for the common core version is capped at 30 people. 

The engineering cornerstone version aims at accommodating up to 200 
first-year engineering students each academic year and is thus offered in 
all four terms. 

This assessment showcase focuses on the common core version 
(ENGGI200) of the Airship project. To run this course, a teaching team 
is formed consisting of five university faculty members in engineering, one 
computer science instructor, two course facilitators (teaching staff), and 
a group of (4-14) student technical advisors (STA). Each student team 
is given a small budget (350-500 HKD) to purchase additional material 
supplies for building an airship within specified design parameters. 


Distinctive Features: 


° An exemplar of tri-modal education structuring undergraduate train- 
ing of research, professional engineering, and entrepreneurship for 
future careers (HKUST, 2015); 

e Scenario-based and mission-oriented tasks (e.g. island rescue, aerial 
firefighting) for the performance of airships designed to enhance stu- 
dents’ learning experiences; 

e Explicitly and comprehensively assessing teamwork skills with an on- 
line toolset, instructor feedback, logbooks and self-reflection. 


Designed Learning Outcomes (LO): 


I Analyse and describe the design of direct-current and logic circuits as 
used in the operation of a microcontroller and create programs in Java 
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for use on Android devices starting with increasing complexity and 
for use in the class project 

II Describe, calculate, and quantify the action of forces and other phys- 
ical quantities on systems similar to that used in the class project 

III Apply an engineering design approach to generate ideas, model, an- 
alyse, predict and build an innovative object of engineering interest 
taking into consideration both societal and economic impact 

IV Demonstrate appropriate knowledge and behaviour for effective and 
ethical membership on a technical team 

V Communicate effectively with others orally, in writing and by use of 
sketches/drawings 

VI Explore possible innovative engineering solutions via peer learning 
and self-initiated learning processes 


Coursework Teaching & Learning Activities: 


Classes 


e Course introduction and project briefing [4 hours; Week 1-2] 

e Effective teamwork, team skills and team building [1-1.5 hours; Week 2] 

e Design processes and creativity [1 hour; Week 2] 

e Technical concepts in computer science, electronic engineering, and me- 
chanical engineering; training on airship construction, mobile phone 
programming and hardware communication [18 hours; Week 3-7] 


Labs - Three sets of lab activities on electronic engineering, mechanical 
engineering and computer science respectively [9 hours] 


Design Studio & Demonstrations 


° Teamwork prototyping, testing and airship building [outside of class 
hours] 

° Application and prototype demonstration [3 hours; Week 8] 

e First build — control demonstration with airship inspection and tasks 
of airship manoeuvres [6 hours; Week 9-10] 


Final Competition [4 hours; Week 14 - the day after the last day of classes 
of the term] 


e Airship inspection 
° Completion of tasks 
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Assessment Type 


The assessment is divided into a wide spectrum of graded activities that are 
mostly formative. The summative assessment is quite innovative, in forms of 
final competition, online peer evaluation tool, and reflection documentation. 
The assessment is also split into individual and team-based portions, valuing the 
significance of teamwork. 


Main Assessment Areas 


Teamwork Dynamics (Collaboration Skills): Teamwork training is a core 
component in the airship course. There are clearly set learning outcomes on 
effective teamwork, class hours on team building, teamwork skills and team 
behaviours, explicit grading criteria on teamwork assessment in the course 
rubrics (A9), as well as a standalone assessment approach on teamwork (A8). 
During the class time on teamwork, the goals of teamwork and appropriate 
team behaviours are set out regarding individual accountability (i.e. all team 
members have basic knowledge of technical modules); interdependence (i.e. 
all team members contribute and know what is happening throughout the 
project), and engaging environment (i.e. teammates are to learn from each 
other). Teamwork is being comprehensively assessed with an online toolset 
(CATME), instructor feedback, meeting logs (A7), and self-reflection. 

The CATME is “a web-based instrument that efficiently collects and 
analyses self- and peer-evaluation data... (which) uses a behaviourally an- 
chored rating scale to measure team-member contributions” in nine areas 
(i.e. Contributing to Work, Interacting with Teammates, Keeping Team 
on Track, Expecting Quality, Having Knowledge/Skills, Team Conflict, 
Team Satisfaction, Team Interdependence, Team Cohesiveness); the “be- 
haviourally anchored rating scale” has been rigorously developed with 
“critical incident methodology” based on team effectiveness literatures 
and team-building experiences (Ohland et al., 2012). The HKUST teach- 
ing team employs this CATME tool for students’ self-evaluation and peer 
evaluation to measure the “team chemistry” from the airship team build- 
ing process (Teacher Interview, Dec 2017). 

Design Thinking (Creativity Skills): The notion and practices of the en- 
gineering design process are weaved into the airship course, involving lab 
making, quick prototyping, design thinking, and essentially, creativity 
skills. The course learning outcomes have specified (i) design in electronic 
engineering, i.e. to analyse and describe the design of circuits; (ii) design 
in computer science, i.e. to conduct Java programming; and (iii) design 
in mechanical engineering, i.e. to describe, quantify and calculate the ac- 
tions of forces with a particular design of airship assembly methods. The 
engineering design approach is applied to generate ideas, model, analyse, 
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predict and build an innovative object. The accomplishment of these out- 
comes is assessed by the design drawings (A3 & A5) as well as the per- 
formance of the final airship project (A9). Three drawings (A3 & A5) are 
required which measure and document the progress of design thinking 
and peer learning process. Each student team needs to submit a Design 
Report (A9) along with the Final Competition. The safety and efficiency 
of the airship design are included in the assessment criteria. Students are 
encouraged to use the least resources possible in building the airship that 
meets the design parameters, with bonus points for the lowest budget 
(A6; A9). Students work on designing a reasonable but creative assembly 
method of available devices (A9). 

Visual Communication: Visual communication with computer-aided de- 
sign (CAD) software is an essential skill for professional engineers and 
design engineers to explore design ideas, to visualize concepts through 
photorealistic renderings, and to project real-world simulations. Students 
in the airship course practice visual communication by drafting their air- 
ship ideas with a free starter CAD software. Their drawings need to be 
three-dimensional and indicate the front, side and top views of the airship 
labelled with measurement units (A3& A5). 

Self-management with Tri-modal Education: For the transition into the 
four-year undergraduate curriculum, HKUST has implemented the tri- 
modal education that *provides students with a career aspiration track 
system based on what they want to achieve? (SENG, 2014). This curricu- 
lum is “designed to increase students’ opportunities to experiment, help- 
ing them to resist conservatism and take advantage of internationalism” 
with “cross-disciplinary courses, active learning, and in-depth academic 
engagement” (HKUST, 2015; UGC, 2015). The “active learning provi- 
sion” is being extended with the development of “innovative practicum 
or experiential-based courses” (HKUST, 2015). Departments in engi- 
neering work with the School of Engineering to provide platforms for 
students on three different paths — research, professional engineering, 
or entrepreneurship. Three of the departments work together to deliver 
the Airship course as part of the tri-modal education, *which allows stu- 
dents to ‘feel’ like an engineer’ (SENG, 2014). The tri-modal design of 
the airship course enhances the self-management skills of undergraduate 
students (especially freshmen). As the tri-modal design intends to be “as 
individually accommodating for students as possible" (UGC, 2015), stu- 
dents should develop a strong self-understanding of needs, interests and 
strengths to prepare for active learning. Some assessments in the airship 
project take into account the self-management skills of students using a 
self-understanding survey for team assignment (Al) and self-evaluation 
in team performance (A8). 


Reflection: Reporting on learning reflection constitutes one of the as- 
sessment techniques for teamwork evaluation (A8). Students are asked to 
thoughtfully and thoroughly respond to several probes for reflection, in- 
cluding: to describe one’s evaluations of team experience; to document 
engineering design process (design decisions); and to document how one’s 
taught/learned from teammates. Writing about teamwork experiences 
substantiates students’ knowledge for effective team performance and 
consolidates the pertinent behaviours. Conversely, reflection and debrief- 
ing sessions are arranged during the week after the final competition (A9) 


Assessment Cases around the World 237 


(Teacher Interview, Dec 2017). 


Assessment Standards/ Sample Rubrics 


Indicators Excellent — Good — Satisfactory — Poor — 
Level 4 Level 3 Level 2 Level 1 
Precision & Drawings are Drawings are Drawings are Drawings are 
» clarity completely mostly labelled ` somewhat labelled not labelled with 
2 labelled with with dimensions with dimensions dimensions and 
z dimensions and and units. and units. units. 
A units. 
w Perspective Front, side and Front, sideand Front, side or Front, side or 
x top views of top views of top views of top views of 
the airship are ` the airship are the airship are the airship are 
included. included. missing. missing. 
Teammate All teammates Most teammates Half of the One individual 
£, contribution provide provide teammates is responsible 
E significant significant provide significant for most of the 
m contributions to contributions to contributions to project. 
8 project. project. project. 
= Progress & Progress and Some progress Gaps in the No progress or 
w evaluation work evaluation and work progress and work work evaluation 
< is clearly evaluation are evaluation exist. is shown. 
demonstrated. | demonstrated. 
2 Completion & All lab activity Mosr lab activity Few lab activity Lab activities 
vd accuracy components components components are poorly/ not 
x are completed are completed are completed completed. 
3 correctly. correctly. correctly. 
Peer Peer evaluation Peer evaluation Peer evaluation is Peer evaluation 
5 evaluation is completed. is completed. completed. is not completed. 
S Reflection All reflection Most reflection ` Reflection Reflection 
S questions questions questions questions questions 
Ex are answered are answered are answered are answered 
Z thoughtfully thoughtfully and superficially. superficially. 
and thoroughly. thoroughly. 
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Teacher’s Stories 


Prof. Ben Chan 


Professional Engagements 


Ben is the director at the Center for Engineering Education Innovation at 
HKUST. His role at the school is specialized for the needs of expanding 
school-based course offerings and student supports along with the four- 
year undergraduate curriculum reform. The airship courses are among the 
ten courses recurrently delivered by this Centre. 


Motivation 


Ben believes that students are already receiving qualitative education on 
cognitive knowledge, but there is a gap in students’ training on holistic 
competencies and positive attitudes (i.e. ethics and professional identity). 
Thus, the common core version of the airship course focuses on students’ 
creativity, teamwork and communication, instead of their technical skills. 
Meanwhile, Ben does not want to overwhelm students with excessive as- 
sessments and selects only one or two quantifiable items of learning out- 
comes to assess (Teacher Interview, December 2017). 


Challenges 


In terms of how to integrate the assessment of holistic competencies with 
curriculum, challenges are shortage of pertinent expertise, limited soci- 
etal recognitions and insufficient industrial accreditations. The expertise 
within the faculty members lies mostly in technical disciplines. There is no 
expert to deliver contents on holistic competencies and no one knows how 
to assess them appropriately, without any expert professor on teamwork 
or communication within the school of engineering. Moreover, there is 
insufficient coverage on holistic competencies in the graduate attributes 
identified by the local professional accreditation agency in engineering 
(i.e. Hong Kong Institution of Engineers). The university needs to make 
society appreciate training on holistic competencies and institute fair 
judgements and recognised assessments to gain buy-ins from the industry. 
However, resources or knowledge within university settings is largely in- 
adequate to accomplish qualitative assessments on generic competencies. 
(Teacher Interview, December 2017) 
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Students’ Side of Stories 


e°  "[t wasn’t tiring because I wanted to do it.” “Yeah, we had fun!” “Even 
though none of us had any experience of building an airship or a ro- 
bot, we believed we were not inferior to the other more experienced 
teams.”...“I have a good team. Thank you!” (Team members of the 
Troublemakers, winning team of the 2014 Airship Final Competition) 

e “The competition provided a unique opportunity to experience an 
engineer’s life. The students gave numerous examples of the similar- 
ities between the two: building something from zero, finding opti- 
mum solutions, facing unpredictability, and sometimes sacrificing rest 
time to do something they are passionate about.” (Fong, 2014) 

e “For the past two semesters, I've been a student technical adviser for a 
total of about 90 university level students, helping out with the tech- 
nicalities of running and developing the ENGG2990D (soon to be- 
come ENGG1200) Airship design course” (Ng, STA, blog 2015) ... 

e “I have taken the responsibility of aiding faculty to developing the 
three credit, undergraduate, engineering team airship design course ... 
I take lead within a small group of student technical advisers and 
complete the following: create, organize and distribute the necessary 
course content that faculty have left out; create the assignments that 
go with the course; create the course assessment criterion; implement 
new and fresh ideas for the course. In the brief time that I have been in 
this position, I have implemented a drawing module, which is essential 
for clarity in the conveying designs” (Ng, LinkedIn, STA 2014-15). 


Featured Videos 


Group 2015 version of ENGGI200: https://Avww.youtube.com/watch?v= 
LgJi_3f8Ms0 

Final Airship Competition (Spring Term 2015/16): https://www.facebook. 
com /airshipdesign /videos/1059613184134342/ 

Final Airship Competition (Fall Term 2016/17) https://www.facebook. 
com /airshipdesign /videos/1228947317200927/ 

Final Airship Competition (Fall Term 2014/15) https://video.ust.hk/ 
Watch.aspx?Video=6E59A6A75E4E5D1F 

Airship engagements with secondary school students: https://www.you- 
tube.com/watch?v=iwVOwyqNeO4 

Engineering Summer Camp HKDSE students: https://www.youtube. 
com/watch?time_continue=2&v=sfqSNQco5bg 
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8.3 Case 3: “Hands-on Work Experience — Assessing 
Practicum in Health Sciences,” Simon Fraser 
University, Canada 


The practicum (HSCI 880) is a compulsory 11-week module in the sum- 
mer term with hands-on work experience for postgraduate students pur- 
suing Masters of Public Health (MPH) at Simon Fraser University (SFU) 
in Canada. Students in the practicum learn how to apply public health 
concepts, methods and theories in designated workplaces. They are men- 
tored and supported by qualified public health supervisors and faculty. 
The practicum is offered in various settings of public health practices, 
including policy, education, communities or health services. There are 
four concentrations in MPH (Environmental and Occupational Health, 
Global Health, Population Health Science, and Social Inequities and 
Health). 

Students in the Global Health concentration must do international 
practicum, with previous students being placed in countries like Malawi, 
Zambia, Sierra Leone, South Africa, India, Mongolia, Mexico, Iran, Aus- 
tralia and the United States. MPH students in other concentrations can 
choose to do their practicum in Canada or abroad. 
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Distinctive Features: 


e Fostering global placements particularly in developing countries of 
public health practices 

° Connecting field practices with potential capstone development 

e Handing students initiatives and responsibilities 


Designed Learning Outcomes (LO): 


I Apply relevant theories, concepts, and skills learned through academic 
coursework in a practice setting relevant to their interests in public 
health; 

II Develop confidence in applying specific skills and relevant theory per- 
tinent to their areas of interest, expertise, and practice; 

III Develop an appreciation for public health practice while working in a 
professional environment, through ongoing personal reflection; 

IV Improve planning, organisational, and communication skills; 

V Demonstrate collaborative skills while working with colleagues in a 
professional practice setting. 


Coursework Teaching & Learning Activities: 


Pre-practicum arrangements: travel preparations, development of an ap- 
proved practicum plan, completion of practicum forms, immunisations, 
visas, and setting up living arrangements, etc. 


° Ist meeting with the practicum committee (formed among the stu- 
dent, academic supervisor, and workplace preceptor) 


Practicum field work (a minimum of 11 weeks of full-time work at a des- 
ignated workplace) 


° 2nd meeting with the practicum committee (midway through the 
practicum) 
e 3rd meeting with the practicum committee (end of the practicum) 


Post-practicum poster preparation (2 weeks) 
Practicum Poster Session (all MPH students are expected to attend early 
fall) 
Debrief session: reviewing experiences and future learning needs. 
Source: SFU (2017) Practicum Guide. 
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Assessment Type 


Both formative and summative assessments are employed before, during and 
after the practicum to capture student learning experience. 


Main Assessment Areas 


Holistic Competencies: The practicum experience is “designed to assist 
students in acquiring skills as a team player, problem-solver, and effec- 
tive communicator” while “organizational and critical thinking skills 
are sharpened as a result of this experience” (SFU, 2017). The super- 
vised field learning is competency-based, and students need to conduct 
a self-assessment of core competencies as an initial step of placing practi- 
cum. Self-understanding of holistic competencies is thus enhanced. Stu- 
dents can choose which core competencies they want to focus on during 
the practicum. Alongside the self-elected core competencies identified 
by the students themselves, the assessment form (A4) to be completed 
by the internship preceptor covers four areas of holistic competencies, 
including interpersonal communication, professional conduct, effi- 
ciency at work, and teamwork. 

Knowledge Application: The practicum is an opportunity for students to 
apply and use their knowledge acquired through coursework, to deepen 
knowledge in a particular area of expertise, and/or expand their breadth of 
knowledge. It is also a collaborative learning process encouraging knowl- 
edge translation and exchange in the Practicum Plan (Al) and the Bi- 
monthly Reports (A2). 

Reflection: The learning outcome of ongoing personal reflection is high- 
lighted in this course. The Bi-monthly Reports (A2) should be reflective 
in nature, “to facilitate a progressive reflection on the practicum that sup- 
ports the learning experience? (SFU, 2017). The poster (A3) should con- 
tain a section on reflection of lessons learnt or key insights gained from the 
practicum. 


Assessment Standards/Sample Rubrics 


As an MPH program requirement, this credit-bearing practicum is graded as 
CO for complete, S for satisfactory, or U for unsatisfactory. The credit points or 
marks are not relevant if the students complete the field requirements satisfac- 
torily. Therefore, rubrics are not provided for most of the assessment activities, 
although an assessment form (A4) and a template for Practicum Plan (Al) are 
available. Questions in the assessment form are listed as follows: 
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Teacher’s Stories 


While the practicum is coordinated and administered by managerial staff 
in the MPH program, the practicum design and arrangements are very 
much a group effort of the entire faculty of health sciences. 


° Prof. John O Neil was the former Dean of Faculty of Health Sciences 
at Simon Fraser University. He comments that 


Probably the most important part of their (MPH) training is the 
practicum, and all of our students do a 12-week practicum in a public 
health or population health setting. ... Our students receive a broad 
interdisciplinary training and they take with them into the field and 
into their careers, I think one of the strongest backgrounds in public 
health and global health than any groups of students in the world. 
(Online video, Sept 2010) 


e Prof. Craig Janes, emeritus and former Associate Dean of Health 
Sciences at Simon Fraser University, says that: 


Mongolia is a very interesting place for students to come to study. 
It represents in many ways a microcosm of globalization. It's a very 
rapidly developing society, with many of the problems that developing 
societies have. At the same time it also has a merging epidemics of 
non-communicable diseases which are very similar to what developed 
countries have. So it's a very good place for students to come to study. 
Our students are able to take advantages of very unique placements. 
They learnt a great deal about global health. 

(Online video, Sept 2010) 


Professional Engagements 


Before joining SFU, Dr. John O'Neil was Director of the Manitoba First 
Nations Centre for Aboriginal Health Research. He has been appointed as 
a Senior Investigator at the Canadian Institutes of Health Research and as 
the founding Chair of the Advisory Board for the Institute for Aboriginal 
People's Health at the Canadian Institutes for Health Research from 2000 
to 2006. He also served as the research advisor to the health policy team 
for the Royal Commission on Aboriginal Peoples in 1995/1996 and he is 
currently appointed to the Advisory Board of the National Collaborating 
Centre on Aboriginal Health at the Public Health Agency of Canada. Dr. 
Craig Janes, when he was a faculty member at SFU, received the National 
Medal of Honour from the Government of Mongolia for his work to de- 
velop the country's health sector. 
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Partnership 


° Oyun Lkhagvasuren, Director of the Health Promotion Division in 
the Department of Health in Mongolia, is one of the global partners 
and workplace preceptors of the practicum. She states that: ^We like 
our collaboration with Simon Fraser, because it's beneficial for both 
sides, for the Canadians and Mongolians, and we are putting together 
our efforts to address the issues that we have in Mongolian terms of 
public health" (Online video, Sept 2010). 

e  Tsogtbaatar Byambaa, Ministry of Health & Global Fund in Mon- 
golia, Global Fund support is the largest of its kind in Mongolia. 
The money was there, efforts were there but our capacities weren't 
good enough, and SFU practicum students came and helped us to 
strengthen our capacities, and benefits were mutual (Online video, 
Sept 2010). 


Students? Side of Stories 


e “J felt the need to gain knowledge that would help me work on a 
much larger scale to reduce health inequalities and improve access 
to better healthcare services in developing countries like mine and 
the world at large. ... While in the MPH program, I have learned so 
much from the wide range of courses that I have been exposed to, and 
I have been able to narrow my interests to the area of Maternal and 
child health. The highlight of the program has been the practicum ex- 
perience where I had the opportunity to work in my area of interest" 
(MPH student from Nigeria). 

e “Today I am here in this secondary school to help evaluate an HIV 
aids trainer program and peer education program. This summer I am 
going to be working with the Department of Health, to help evaluate 
this program that has been funded by the Global Fund. But I decided 
to come to Mongolia based on the previous experiences as former stu- 
dent here and the relationship that they form as well as the experience 
of Mongolian culture" (MPH student doing practicum in Mongolia, 
Online video, Sept 2010). 

e “At first, I have had the experience of what is [sic] successful pub- 
lic health intervention can look like, and so we have a lot of learn- 
ing about challenges and struggles, and some projects that haven’t 
worked out and so to have the real experiences of what the changes 
can mean for vulnerable populations ... was really motivating, it was 
really inspiring to know that public health projects can work and can 
have a real impact” (MPH graduate, online video, Sept 2010). 
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e “Through all the courses we have taken in this program and through 
our experiences abroad in our internships, we are taught to take into 
account and to give importance to the social context, or political 
context, economic context, and how those relate into health out- 
comes and health data of populations” (MPH graduate, online video, 
Sept 2010). 

e “When I was a student in the master program at SFU, I did my practi- 
cum placement at the Department of Health (of Mongolia), and I really 
learnt through that experience about the complexity and importance of 
working with various stakeholders in the public and private and NGO 
sectors, and these are skills that I think I would take with me as I con- 
tinue my career” (MPH graduate, online video, Sept 2010). 
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8.4 Case 4: “Understanding British Legal System — 
Assessing Pre-law Track Student Performance in the 
Law Course of London Internship Programme”, 
Boston University, the United States of America 


This course (CAS PO 22) is part of the Boston University London In- 
ternship Programme, in which students complete both credit classes and 
internship/work placement in London for 15 weeks (one semester) in 
their chosen fields of interest. This course is mandatory for Pre-Law track 
students in the Internship Programme. With a combination of lectures, 
seminars, field trips and self-reading, students are able to gain an over- 
view of the history and the development of the British legal system, and 
thoroughly examine its contemporary operation and iteration. Students 
are also expected to learn about the legal profession, the judiciary and the 
legislature of Britain, and the massive influence of the English common 
law/legal system on the American common law/legal system. 
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Distinctive Features: 


e Featuring a wide variety of learning activities including lectures, dis- 
cussions and field trips 

e Relating the British legal system to the American legal system to pro- 
vide the foundation for an in-depth study of the legal world 

e Offering a thorough overview of the British legal system on both the- 
oretical and practical aspects 


Designed Learning Outcomes (LO) 


I To gain a basic understanding of the origins and developments of the 
common law, the English legal system of courts and the criminal and 
civil litigation processes 

II To examine the development of as well as the current British legal 
profession, including routes to qualification, operating vocabulary, 
current legal trends and local legal peculiarities, and how access to 
justice is funded in the British legal system 

III To understand, examine and critique the contemporary form of the 
judiciary and the legislature. 


Coursework Teaching & Learning Activities 


e Lectures (8 sessions): Introduction to different aspects of the English 
legal system 

e Class Discussion: Students are expected to read newspapers daily and 
select articles relevant to the course. They might be required to sum- 
marize the article to the class and lead a class discussion on the subject 
matter 

e Field Trip 1 (Week 1): Orientation field trip 

e Field Trip 2 (Week 2): “Legal London Walk” including a visit to the 
Royal Courts of Justice 

e Field Trip 3 (Week 4): Field trip to Houses of Parliament 

e Field Trip 4 (Week 6): Field trip to Crown Court, Old Bailey 

e Field Trip 5 (Week 7): Field trip to Magistrates Court 

Source: Boston University (2016) Course Syllabus 
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Assessment Approaches 


Name 


Learning Weight 


Activities 


Aligned Type 


LO 


Generic Skills 


Al Attendance 


A2 Final 
Examination 


Attendance is 096 
compulsory in this course (penalty 
for all class sessions, applies) 
seminars and field trips. 

Students absent for more 

than two class sessions 

(whether authorized 

or unauthorized) will 

be required to meet 

with the Directors of 

the programme. An 

unauthorized absence 

will lead to a 4% grade 

penalty on the students’ 

final grade. Class 

attendance will also be 

used in the moderation 

of final grades. 


Students are required 40% 
to take a 2-hour closed 

book examination at 

the end of the course. 

They will need to answer 

two questions from a 

list of four. Students are 
expected to answer each 
question completely with 

3.5 to 4.5 pages in length. 


IL 
IL 
TIT: 


I. 
We 
IDUL 


Formative 


Summative 


Self- 
management; 
Responsibility 


Critical 
Thinking; 

Study Skills; 
Communication; 
Time 
Management; 
Problem Solving 


A3 Essay 


All students are given 60% 
the same topic for the 
essay and are required 

to produce a 2,500- 

word response to the 
essay prompt. They 

are encouraged to use 
illustrative materials 
including texts, 
periodicals, press or other 
media. 


Assessment Type 


JUL. 


Summative 


Critical 
Thinking; 

Study Skill; 
Communication 


Both summative and formative assessment methods are adopted in this course to 
enhance student engagement and learning from the course. 
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Main Assessment Areas 


Holistic Competencies: With a comprehensive combination of learning 
activities ranging from lectures to field trips, this course focuses on devel- 
oping students’ critical thinking and study skills. The final examination 
(A2) and the essay (A3) are both effective assessment methods of holistic 
competencies including critical thinking, study skills and communication 
skills because students are required to critically and eloquently answer 
questions given to them with knowledge learnt in the course. At the same 
time, class attendance (A1) can train students to manage themselves well 
and have a sense of responsibility. 

Knowledge Application: The practical application of knowledge learned 
in this course is another assessment focal area. In the final examination 
(A2) and the essay (A3), students are required to, based on knowledge 
learnt in the course, critically respond to questions related to areas dis- 
cussed. This assessment focus is in line with the course LOs, which focus 
mainly on the understanding and grasping of new knowledge. 


Assessment Standards/Sample Rubrics 


A set of general grading criteria is set out for the whole course, while some of the 
assessment approaches have other specific criteria. 


Description Grade 


General Grading Criteria 


This exceptional grade is assigned only to work that has persistently A 
outstanding quality in both substance and presentation. The student 

must demonstrate a sustained capacity for independent thought and 

extensive study, producing rigorous and convincing analyses in well- 

ordered prose. 


Awarded to work that is clearly focused and analytical, and based on A- 
wide reading. The student must cover all the principal points of a 

question and systemically develop a persuasive overall thesis, allowing 

for one or two venial omissions or inapt expressions. 


This range of grades indicates that the student has shown some B+, 
evidence of original thought and intellectual initiative. The student B, 
has cited sources beyond the class materials and shown a degree of B= 


originality in perception and/or approach to the subject. 

The work will show thoughtful management of material, and a good 
grasp of the issues. 

The differences between a B+, a straight B and a B- may reflect poor 
presentation of the material, or mistakes in punctuation, spelling and 
grammar. 
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Description Grade 
Work in this grade range is satisfactory, but uninspiring. If the work C+, 
is simply a recitation of the class materials or discussions and shows e? 

no sign of genuine intellectual engagement with the issues, it cannot C- 


deserve a higher grade. Should an essay fail to provide a clear answer 
to the question as set, or argue a position coherently, the grade will 
fall within this range. 


A marginal pass can be given where some but not all the elements of D 
the course have been completed satisfactorily. 


The failing grade indicates the work is seriously flawed in one or more E 
ways: 


e Obvious lack of familiarity with the material 

* So poorly written as to defy understanding 

e So brief and insubstantial that it fails to properly address the 
subject 

° Material presented is not relevant to the assignment 

° Demonstrates evidence of plagiarism 


General Grading Criteria 


Source: Boston University (2016) Course Syllabus 


Criteria 


The grade students receive will be based upon: 


* The relevance of your answer to the question/topic set/to be discussed 
* Clarity of expression and continuity 

e Evidence of reading and thought relate to the question/topic 

e 

e 


A2 & A3 


Quality of the arguments presented 
The above Grading Criteria 


Source: Boston University (2016) Course Syllabus 


Teacher's Stories 


Denis Carey 


Professional Engagements 


Dennis Carey holds a Bachelor of Arts degree from the National Univer- 
sity of Ireland, a Post Graduate Diploma in Law from Dublin Institute of 
Technology, a Master of Laws from Georgetown University Law Center 
and a Post Graduate Diploma (Roman Law, Private International Law 
and a lawyer in Ireland, California and England & Wales. He has vast 
experience in legal education and overseas programmes in various institu- 
tions, including Staffordshire University Law School, University of Idaho 
College of Law Summer School and University of Tulsa College of Law's 
London Programme. He has taught various law courses for the Boston 
University British Programmes. 
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Motivation 


Denis is admitted as a lawyer in both England and California. With his 
knowledge and experiences in both the British and American legal systems 
and the organisation of overseas programmes, he is able to deliver the LOs 
with a balance of lectures, discussions and field trips. 


Students’ Side of Stories 


e “In terms of rigor, the courses were much easier than at my home 
institution. However, the classes themselves for the most part were ex- 
tremely fun. We went on a lot of class trips to museums, the Supreme 
Court walking tours, I even had a free trip to Belgium. The professors 
prioritize exposing their students to England’s culture and history. 
Very enjoyable!” (Anonymous student rating, 2017) 


Reference 


Boston University. (2016). The British legal system [Syllabus]. Study Abroad, Boston 
University. Retrieved from https://www.bu.edu/abroad/files/2016/02/CAS-PO- 
222-THE-BRITISH-LEGAL-SYSTEM. pdf (accessed 4 Apr 2018). 


8.5 Case 5: “Building Personal and Workplace Skills — 
Assessing Humanities Internship,” University of 
Otago, New Zealand 


The Humanities Internship Practicum (HUMS301) is an elective for all 
full-time undergraduate students in any of the Humanities disciplines at 
the University of Otago in New Zealand. Students are expected to work at 
a partner organisation for eight hours per week, for 13 weeks. There is no 
automatic admission to the elective, and students must receive approvals 
from both the departmental supervisor and the course coordinator to be 
admitted. This course focuses on enabling students to develop a range of 
personal and interpersonal skills, and it also allows students to learn about 
the reality of the workplace. Students are expected to reflect on their in- 
ternship experiences and make effective use of what they learnt during the 
internship. Through developing personal and workplace skills of students, 
their employability can be enhanced. 


Distinctive Features: 


e Generating and agreeing on expected learning outcomes among the 
student, the university and the partner organisation 


254 Assessment Cases around the World 


I 


II 


IV 


VI 


Assessing students through hosting a Humanities Symposium at the 
University of Otago 

Giving students a choice of assessment formats among written report, 
portfolio and hybrid combination 


Designed Learning Outcomes (LO) 


To develop a range of personal and interpersonal skills, particularly 
communication, team working, priority setting and time manage- 
ment skills 

To demonstrate applying, integrating and evaluating the body of 
knowledge and the method of inquiry of a discipline or a field via 
first-hand participation 

To show the ability to learn the expectations of employers and the 
reality of the workplace 

To present evidence of internship experiences clearly and appropriately 
To reflect on experiences and the learning derived from the internship 
To investigate a topic relevant to the internship context or experience 


Coursework Teaching & Learning Activities 


Pre-Internship 


Discussion with students! department 

Preliminary Project form 

Initial meeting with workplace coordinator and departmental liaison 
person 


During the internship 


Internship and reflective field notes 
Report/portfolio 


Post-Internship 


End-of-placement assessment form 
Symposium 
Source: Humanities Division, University of Otago (2018). 


Assessment Approaches 


Name 


Learning Activities 
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Weight 


Aligned Type 


LO 


Generic Skills 


Al 


Reflective 
Field Notes 


Students have to 
complete a weekly 
diary, approximately 
one page per week, 
containing reflections 
on how their work 

is related to theories 
they have learnt at the 
university. Students 
are expected to 
discuss their reflective 
field notes with 

the departmental 
supervisor at 
fortnightly meetings. 


25% 


Il. 


VI. 


Formative 


Communication, 
Critical Thinking, 
Problem-solving, 
Self-management 


A2 


Final 


report/ 
Portfolio 


Students have to 
submit a final report/ 
portfolio at the end 
of the internship to 
demonstrate their 
ability to undertake 

a research project 
related to their work 
experience 


60% 


I. 
JUL; 
IV. 


Summative 


Critical Thinking, 
Problem-solving, 
Creativity, Study 
Skills 


A3 


End-of- 
placement 
assessment 


The assessment is a 
symposium session for 
students to showcase 
their ability to present 
their internship 
experience and to 
critically reflect 

on their personal 
development of 
different types of skills 
and competencies 


Assessment Type 


1596 


V. 


Summative 


Communication, 
Problem-solving, 
Critical Thinking 


The assessment approaches demonstrate a mixture of formative and summative 


assessments, although the summative ones are the focus. This enables the eval- 


uation of students? growth throughout the internship and of the learning out- 
comes after the internship. 
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Main Assessment Areas 


Problem-solving Skills: Students are expected to manage the chal- 
lenges of the workplace during the internship. With weekly field notes 
and fortnightly discussions with their supervisor (A1), students receive 
advice on their problems and suggestions for improvement. The final 
report/portfolio (A2) and end-of-placement assessment (A3) then al- 
low students to reflect on their problem-solving skills throughout the 
internship, which also test students’ problem-solving skills as students 
may meet challenges while presenting their internship experience in 
written and verbal forms. 

Communication Skills: Throughout the internship, students are expected 
to communicate with people in the workplace and with supervisors both 
in the workplace and at the university. In discussions with the supervisor 
(A1), students need to communicate effectively to show what they have 
done in the internship. Students are expected to actively seek feedback 
and suggestions for improvement during such discussions. For the end- 
of-placement assessment (A3), students must outline and discuss their 
internship experience through a verbal presentation. Students also must 
communicate with the audience and answer any questions the audience 
may have. Communication is thus assessed in various ways. 

Knowledge Application: Students are expected to apply acquired knowl- 
edge during their internship. This can be assessed by the supervisor 
who has regular discussions with a student on workplace skills and 
challenges (Al). Through writing weekly reflective field notes (A1), 
students are encouraged to reflect on how their work is related to the- 
ories taught in the university, applying knowledge they have learnt in 
lessons to their workplace experience. Students can also employ the- 
ories or concepts in analysing the problems they perceive during the 
internship, in the final report/portfolio (A2) and the end-of-placement 
assessment (A3). 

Values for Reflection: Reflection on job performance during the in- 
ternship is assessed in the weekly field notes (A1), whereas fortnightly 
discussions with the supervisor (A1) may enhance the reflection process 
and provide more insight as to how the student can improve. Reflection 
on the development of different skills and achievements throughout 
the internship is suggested in the final report/portfolio (A2) and the 
end-of-placement assessment (A3). The end-of-placement assessment 
(A3) is also a way to push students to further reflect upon what they 
have learnt and have acquired from the internship experience, as there 
is discussion time for attendees of the symposium to raise questions on 
the presentations. 
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Teacher’s Stories 


Prof. Claire Freeman 


Professional Engagements 


Prof. Freeman was a planning coordinator for the Urban Wildlife Trust for 
the West Midlands and was a Senior Lecturer for Planning and Housing 
Professional Group at Leeds Metropolitan University from 1991 to 1997. 
She then became a Senior Lecturer at the School of Regional and Re- 
source Planning, Massey University from 1997 to 1999. Currently a pro- 
fessor at the University of Otago, she is mainly in the Master of Planning 
program with teaching involvement in Planning Theory, Spatial Planning 
and Development, and Internship Practicum. The research interests of 
Prof. Freeman lie in planning and development of the natural environ- 
ment, planning with children, and sustainable settlements. 


Students’ Side of Stories 


° “T really enjoyed HUMS301 because they're more hands on and I 
was able to do something that I was passionate about [...] This pa- 
per bridges a gap between being a student and we'll go into the real 
world, and having a mentor who is able to guide you from what you 
have learnt within your undergraduate degree to what and how you 
apply them in work situation...” (Student S, Nov 2016). 


Featured Video 


e Reflection of a student who has completed the internship https://youtu.be/ 
TZeMgqlXcd5k 


References 


Course website. Retrieved from http://www.otago.ac.nz/humanities/study/otago611002. 
html#what 

The University of Otago. (2018). Humanities division HUMS 301-401 internship 
practicum programme manual. Retrieved March 22, 2018, from http://www.otago. 
ac.nz/humanities/otago611269.pdf 
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8.6 Case 6: “Practicing Geographical Fieldwork Skills — 
Assessing a Field-Intensive Geography Class,” 
University of Western Ontario, Canada 


The course Field Methods and Practices (Geog 3000y) at the Univer- 
sity of Western Ontario (2022) involves seminars and a one-week field- 
intensive excursion. It is open to third-year students in the Department of 
Geography in the Fall semester each year. At an unfamiliar location away 
from the university campus, students are given the opportunity to practice 
geographical fieldwork skills through conducting authentic geographical 
research. As both the preparation prior to departure and the on-site prac- 
tice of geography are equally important, students are required to attend 
seminars and complete assignments before, during and after the field trip. 

There are two locations available: Montreal (Canada) and Kentucky (the 
United States). This case study investigates the Montreal course. 


Distinctive Features 


e Featuring a combination of seminars and field trip, along with a vari- 
ety of assignments prior to, during and after the field trip 

e Providing an opportunity to conduct geographical research at an un- 
familiar location 

e  Stressing on the values of learning by doing hands-on fieldwork 
exercises 


Designed Learning Outcomes (LO) 


I To develop students’ practical geographical research skills, including 
field-based observation, data collection and recording, and analysis 
and interpretation of human and physical landscapes 

II To develop observation, analytical, critical, personal and group skills 

III To facilitate student-centred learning and experiential learning with 
authentic field research 

IV To foster respect for the urban environment 


Coursework Teaching & Learning Activities 


° Seminars (3 sessions, 1°t session: 2 hours; 2" and 34 session: 3 hours): 
1 session is held prior to departure and 2 sessions are held after the 
trip to supplement learning from field work 

e Seminar Meeting (1.5 hours, Day 2): a talk on field note booking, 
lecture and group allocation 
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e Agenda Meetings (1/ 1.5 hours, Day 3—5): lectures and discussion of 
field assignments 

e Guided Tour 1: Old & New Montreal Tour with Urban Specialist 
(Day 2) 

e Guided Tour 2: Tour of Old City Hall (Day 4) 

e Guided Tour 3: Museum of Archaeology & History (Day 5); 

e Completion of Field Assignments (Day 2-5) 

e Discussion Sessions (2 sessions, lst session: 1 hour; 2nd session: 0.5- 
hour, Day 3, 5) 

Source: The University of Western Ontario (2022) 


Course Syllabus. 


Assessment Approaches 
Name Learning Activities Weight Aligned LO Type Generic Skills 
Al Pre-trip Students are required 10% Il. Formative Study Skills; 
Assignment to research the different III. Critical Thinking 
aspects of Montreal IV. 
(e.g. city, people and 
culture) and submit 
the assignment before 
departing for the field 
trip. 

A2 Field Students are asked to 40% I. Formative Critical 
Notebook & keep records of their Ie Thinking; Self- 
Assignments observations and Il. management; 

experience during the IV. Problem Solving 
field trip, including 


all notes from lectures 
and presentations, 

and complete field 
assignments in their 
field notebooks. The 
field notebooks are 
submitted at the end of 


A3 Research 
Proposal 


the field trip. 

Students are required 40% Il. 
to complete a ID. 
2,000-word research III. 


proposal developing 
their own research 
question relevant to 
themes covered in the 
field trip. Students 
are encouraged to 
discuss their research 
topics with the course 
instructor during or 
after the field trip. 


Formative/ Communication; 

Summative Critical 
Thinking; Study 
Skills 
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Name Learning Activities Weight Aligned LO Type Generic Skills 
A4 Participation Students are 10% Vl, Formative Self-management; 
expected to attend IV. Responsibility: 
and be punctual Communication 


for all seminars and 
meetings. Their 
attitudes to instructors 
and guest speakers and 
Participation in class 
will also be assessed. 


Assessment Type 


A number of formative assessments are adopted in this course to enhance student 
engagement in the field trip and ensure the quality of student learning through 
the trip. A summative assessment (A3) is also employed to encourage students to 
research broader urban geographic issues using the field trip as a starting point. 


Main Assessment Areas 


Holistic Competencies. This Geography course emphasizes student- 
centred learning and learning by doing. Hence, developing students’ 
competencies of self-management and critical thinking are the two foci 
in the course. The course’s LO also specifies the importance of enhanc- 
ing students’ observation, analytical, critical, personal and group skills. 
These holistic competencies are assessed in the field notebook and field 
assignments (A2), the research proposal (A3) and participation (A4). In 
the assessment of participation (A4), students are encouraged to actively 
engage in lectures, presentations, tours and discussion sessions, in which 
their communication skills are also trained and assessed. 

Knowledge Application and Geographical Research Skills: In this course, 
research-oriented learning is accompanied by seminars. During seminars, 
students gain knowledge of urban geography and the field site, which they 
can use to analyse and explain urban landscapes and problems. Students 
also learn about practical research skills and practice them in the field. 
The pre-trip assignment (Al) and the research proposal (A3) are both 
assessments of students’ knowledge of urban geography, while the field 
notebooks and assignments (A2) assess students’ geographical skills. 
Reflection: Students are required to prepare and compile their own field 
notebooks (A2), which contain a documentation of their field experiences 
and observations. The completion of field notebooks and field assignments 
can facilitate students’ self-reflection during and after the field trip. In the 
research proposal (A3), students are also asked to reflect upon their field ex- 
periences and observations and develop their own research questions. Lastly, 
students are encouraged to pose questions and contribute to discussions to 
gain participation marks (A4), which can stimulate reflective discussion. 
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Assessment Standards/Sample Rubrics 


A2 Field 
Notebook 


Source 


Criterion 


Format and organisation 
Completion of all assignments 
Detail, quality and originality of the observations 


e 
e 
e 
e Evidence of critical analysis (questions, ideas and conclusions) 


: The University of Western Ontario (2022) Course Syllabus. 


Description 


Marks 


You have performed satisfactorily in the field, completed all aspects 
of the assignments adequately, and demonstrated you have a basic 
grasp of the material. 


C 
(60%-69%) 


General Grading 


An average to slightly above average performance and evidence 
of independent thought, effort and creativity. Not only have you 
fulfilled the requirements of the field work and assignments in a 
more meticulous fashion, but you have also demonstrated a more 
thorough understanding of the material and you have gone a few 
extra steps beyond the average student. 


Your special work exhibits a fastidious eye for detail and clear signs 
of outstanding effort, presentation, content and clarity. 


B 
(70%-79%) 


A 
(80%-89%) 


An exceptionally outstanding set of insightful field notes, a 
research paper demonstrating a superb command of English and a 
critical mind, and positive participation in the course will earn this 
prestigious grade. 


A+ 
(90%+) 


Source 


: The University of Western Ontario (2022) Course Syllabus. 


Teacher’s Stories 


Dr. Jeff Hopkins 


Professional Engagements 


Jeff Hopkins holds a Master of Arts from the University of Western On- 
tario and a PhD in human geography from McGill University. He is an 
Associate Professor as well as the Graduate Chair in the Department of 
Geography at the University of Western Ontario. He was recognized by 
the University Student Council with the Western Teaching Honour Roll 
seven times and was awarded the Social Science Dean’s Award for Excel- 
lence in Teaching at the University of Western Ontario. 
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Motivation 


Dr. Hopkins has dedicated to improving course designs and curricula with 
innovative and engaging approaches. He is also highly recognized for his 
contributions to instructional development and education outreach. This 
course is a very good illustration of Dr. Hopkins’ endeavour in adopt- 
ing engaging pedagogical methods and assessments in the teaching of 


geography. 


Students’ Side of Stories 


e “Best course in university” (Anonymous student testimonial, n.d.); 

° “J learned so much more than I would have in a classroom” (Anony- 
mous student testimonial, n.d.); 

e “Our field work expeditions were the highlight of my academic expe- 
rience" (Anonymous student testimonial, n.d.). 


References 


The University of Western Ontario. (2016). Jeff Hopkins- Excellence award. Retrieved 
on March 29, 2018, from http:;//www.ssc.uwo.ca/news/2016newsarticles/awardex- 
chopkins.html 

The University of Western Ontario. (2017). Field methods and practices: Montreal — its 
urban, cultural & social practices [Syllabus]. Retrieved on July 16, 2022, from https:// 
geoenvironment.uwo.ca/undergraduate/course information/3000 level courses. 
html 


8.7 Case 7: “Being ELITE - Assessing Engineering 
Project-Based Assignments", Chinese University of 
Hong Kong, Hong Kong 


*Social Media and Human Information Interaction" is an intermediate 
course in information engineering, suitable for third or fourth year stu- 
dents in a four-year curriculum. It is an elective course for students ma- 
joring or minoring in information engineering, or engineering students 
in the ‘ELITE Stream’, namely ‘Engineering Leadership, Innovation, 
Technology and Entrepreneurship Stream’ — an innovation in teaching 
and learning devised by the Faculty of Engineering since 2014/2015 to 
strengthen students’ holistic leadership for diverse development. The spe- 
cialist content of the course recognizes social media as one of the main 
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sources of big data and examines the social and human dimensions of so- 
cial media. The course consists of mainly in-class lectures and web-based 
learning, while the students carry out project works via experiential learn- 
ing. The lectures cover the foundations of social media, human cognition 
and information behaviour, online communities and social interactions, as 
well as infographic and big data visualisation. There is a course instructor/ 
coordinator, assisted by three tutors to facilitate the teaching and assess- 
ments of the course. As the ELITE status is offered to students with excel- 
lent academic performance, this course as one of the engineering courses 
for the ELITE stream is designed with additional subject contents and 
challenging coursework assessments. 


Distinctive Features 


e Relating engineering professionalism with global citizenship for ho- 
listic leadership training 

e Integrating human information interaction with the analytics and vis- 
ualisation of big data 

e Specialising in user-application interfaces at the research-teaching 
nexus of data science 


Designed Learning Outcomes (LO) 


I Gain conceptual knowledge and theoretical foundations in social me- 

dia and human information interactions 

II Ability to demonstrate the integration of acquired skills, capabilities 
and knowledge from university studies to a practical workplace pro- 
ject. Ability to demonstrate knowledge of corporate employability 
acquired within a workplace. Professional presentation of the applica- 
tion of theory into practice 

III Capacity to perform project-based work assignments and provide a 
project précis to the workplace 


Coursework Teaching & Learning Activities 


e Lectures [46 hours; Week 1-14] 
e Web-based learning classes [4 hours; Week 7 & 8] 


e Blogging 
° Group work 
° Exam 


Source: Interview with the teacher 
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Assessment Type 


A wide range of summative assessments are employed to assess students’ grasps 
of knowledge. Formative assessment is adopted to encourage student participa- 
tion in making active responses in the online learning community. 


Main Assessment Areas 


Interdisciplinary Teamwork (Collaboration Skills): Students are re- 
quired to work in groups to work on an interdisciplinary project regarding 
the Sustainable Development Goals (SDGs) set out by the United Na- 
tions. This group project encourages engineering students to bring in 
knowledge from different disciplines, especially the theories and concepts 
that they are exposed to in their general education curriculum or minor 
program courses, e.g. social sciences and humanities. This research pro- 
ject is thus interdisciplinary in nature. The course instructors assess the 
output of this interdisciplinary teamwork research based on the relevance 
of recommended solutions to the identified problems (A3). The process 
of teamwork is assessed in another activity — online learning community 
participation (A2). Since each student must create a blog and share some 
learning reflections (A1), the course coordinator encourages students’ col- 
lation of ideas by giving some marks for active comments on blogs of peer 
students (A2). These scores awarded to blog comments are used as incen- 
tives for teamwork. Although rubrics have not been very well developed 
for the blogging and blog commenting activities, students are informed 
about how this work is being evaluated. 

Professional & Ethical Identity of Global Citizenship: Through the 
Group Project One on interdisciplinary topics (A3), “somehow the stu- 
dents can get to recognize their professional identity as an engineer and 
appreciate how important the role that they are going to play, or are cur- 
rently playing, in solving some of the global challenges” (Teacher Inter- 
view, Jan 2018). Developing students’ professionalism and professional 
ethics as future engineers is a special focus of this course. The awareness of 
global issues is being assessed in Group Project One: Policy Report (A3). 

Information Technology (IT) Skills: Students majoring in information 
engineering (or computer science) are expected to master a higher level of 
IT skills than the ones in other academic disciplines. Group Project Two in 
this course assesses students in terms of how they use computer program- 
ming instructions to perform sentiment analysis, how they select com- 
putational tools to visualize data, and how they present analysis results 
(A4). The building of blogs for individual e-Portfolio (A1) also assesses 
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students’ skills in information and communication technology (ICT). To 
conduct the research for Group Project One (A3), students need to access 
databases of the United Nations Statistics Division and identify relevant 
datasets to generate infographics, which assess their information literacy 
skills. 

Knowledge Application: The open lecture-notes format of the Final Exam 
(A5) avoids rote learning and ensures the critical application of theories 
and concepts covered in the course. Group Project Two (A4) examines the 
technical knowledge prescribed by the course. Group Project One (A3) 
requires students to “apply the knowledge in this course and make profes- 
sional recommendations on how ICTs can be applied to solve or mitigate 
the issue...identified” (Project Specification 2017). Students need to jus- 
tify their recommendations with principles in Human-Information Inter- 
action and/or the nature of Information, and some of the core concepts 
covered in the course. 

Reflection & Critical Thinking: The reflection of student learning is as- 
sessed in e-Portfolio (A1), where students reflect on why human informa- 
tion interaction — the subject matter of this course — is important to the 
engineering field. Using blogs to compile an e-Portfolio ensures timely 
reflection of students on the topic they learn along with the schedule of 
lecturers. As blogging is open to the general public, it also ensures stu- 
dents’ understanding of newly acquired knowledge when they need to 
process complex concepts into simple language. The incentivised com- 
menting (A2) on the blogs sustains such reflection on the concepts and 
theories of the course. In Group Project One (A3), students reflect upon 
the roles of engineers and global citizenship, by contemplating solutions 
to global problems. 
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Teacher’s Stories 


Prof. Chan Yuen-Yan, Rosanna 


Professional Engagements 


Rosanna Chan received her BEng, MPhil, MEd and PhD degrees from 
the Chinese University of Hong Kong (CUHK). She served as an assistant 
professor in the Department of Information Engineering, CUHK from 
2006 to 2007, and as a postdoctoral fellow in Strategic Research Theme 
in Sciences of Learning, HKU from 2008 to 2011. Rosanna possesses 
a multidisciplinary background in engineering, education, and learning 
sciences. Her research interests include engineering education, human fac- 
tors in online social networks, and learning sciences. Rosanna founded the 
IEEE Education Society Hong Kong Chapter and is the Founding Chair. 
She is an Adjunct Assistant Professor in the Department of Information 
Engineering at CUHK. 


Motivation 


The need for consciousness and empathy for global issues might be one 
of the motives behind the unique coursework design. Prof. Chan wants 
her engineering students to rethink their professional identity and global 
responsibilities. As Rosanna stated, 


I ask the students to study the United Nations’ Sustainable Devel- 
opment Goals, and to think about how engineering knowledge and 
skills can be applied to those goals; I force my students to think out of 
the box, to work in groups, and try to exercise what they have learnt 
outside of my own course; ...I encourage them to bring in the addi- 
tional knowledge that they (may have been) exposed to... for example, 
some students may take courses from Psychology Department... So 
I ask them to try to propose some engineering solutions to at least 
approach those goals, to make people’s lives easier. 

(Teacher Interview, Jan 2018) 


‘Students’ Side of Stories’ 


e “Whenever we search or download something, our actions will be 
stored as information, as data. These data can be analysed into some 
other useful materials. As engineering students, these materials can 
help us to understand the preference of users. As a result, other than 
suggesting some products to the users, we can design new products 
that are more to their liking. If we can prepare and provide service 
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before the user makes a request, services will be facilitated, and time 
will be saved. Convenience is all we, the engineers aim to achieve” 
(Student A, e-Portfolio, January 2017). 

e “As human information is getting more important nowadays, we 
should know more about it and learn how to process such data and 
use them in the engineering field” (Student B, e-Portfolio, January 
2017). 

e “As engineering students, we have been learning how to design pro- 
grams and circuits. However, it is not enough for us to become good 
engineers. To make a program user friendly, we need to listen to the 
opinion and feedback from users. Then, we need to know how to ob- 
tain useful information from this feedback. Therefore, learning how 
to design programs or circuits — this kind of hard skill is not enough. 
We should also learn how utilize the information we can get in this 
world” (Student C, e-Portfolio, September 2017). 

e “To us, Engineering student, information is a tool for us to change 
the world. We find a way to identify the useful information from this 
world, give out responses and transmit them to others so that we can 
change the actions of others towards our desire. This is the ultimate 
goal as an Engineer" (Student D, e-Portfolio, September 2017). 

° “In my AI module ... the topic has hooked me up and now I am 
wanting to relate my further career to it. However, I realised I have 
very little understanding of fundamental concepts used on the back- 
ground of any AI system — information, which motivated me to take 
up IERG3220. Just after a few classes I am able to structure my knowl- 
edge in regard to some topics that I could only understand intuitively 
before. Starting from the concept of context ..., ending up with topics 
on human cognition ... and how these affect the way we understand 
the input of data ...” (Student E, e-Portfolio, September 2017). 


References 


IERG3320/ESTR3306. Social media and human information interaction — course out- 
line (2017-18 Term 1). 

IERG3320. Social media and human information interaction projects specification — 
project one: Social media and human information interactions from a global citizenship 
perspective. 

IERG3320/ESTR3306. Social media and human information interaction project 
specification — project two: Sentiment analysis. 
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8.8 Case 8: “Experiential Learning in Social Ventures — 


Assessing Social Entrepreneurial Internship”, Business 
School, University of Hong Kong, Hong Kong 


Social Venture Management (SVM) offered at the University of Hong 
Kong (HKU) is an experiential learning course that gives students an op- 
portunity to work directly for social ventures under the guidance of a 
business faculty instructor and professional mentors. Students have op- 
portunities to manage teams, solve real-life problems, and gain hands-on 
business experience. These practical exposures aim to strengthen the ge- 
neric skills of students, help them implement academic business concepts, 
and improve their competency in the job market. Students need to commit 
10 hours per week over one semester to their respective companies. There 
is no class time or final examination. This course is open for students from 
all academic disciplines, including exchange students, thus encouraging 
inter-cultural learning. While local students are eligible for a formal in- 
ternship over the duration of this course, exchange students are given su- 
pervised projects by the instructors pertinent to strategic aspects of SV M's 
partner social enterprises. 

The partner social enterprises of SVM range from pre-incorporated ven- 
tures to more established charities and enterprises. A lot of the new social 
ventures look for students who are highly entrepreneurial and proactive. 
Students can also propose their own business ideas or grow their own 
social enterprises through SVM mentorship. 


Distinctive Features 


e Forming an “impact family,”, i.e. a cohesive network of co-creating 
social impacts through university-led initiatives, with students to 
achieve something meaningful 

e Integrating learning with practices of social entrepreneurship 

e Making use of the process of starting up social ventures as a teaching 
tool for students 


Intended Learning Outcomes (LO) 


I Integrate and apply theoretical knowledge to tackle real business 
challenges 
II Research, analyse, assess and propose practical business options and 
solutions given the project constraints 
III Exchange ideas and experiences with partners, resolve differences, mu- 
tually enhance personal development and accomplish tasks through 
collaboration 
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IV Adjust to a real-life work setting and understand the organisational 
culture of a business 
V Present findings and business solutions (both written and oral) in a 
professional and persuasive manner 
VI Apply the knowledge in service leadership to improve their leadership 
quality and effectiveness 


Coursework Teaching & Learning Activities 
Pre-Internship Preparation 
e  Pre-Internship Orientation (1 hour, Week 1) 


e First Meeting with Social Ventures GM (1 hour, Week 1) 
e Creation of Action Plan (5 hours, Week 1-2) 


During Internship Learning 


e Internship Work and Weekly Reports (120 hours, Week 3-12) 
e Video Assignment (10 hours, Week 3-12) 


Post-Internship Consolidation 


e . Exit Interview (optional; with GMs and Supervisors; 0.5 hours, Exam 
Week) 


As a result of the internship, real-world business activities were organised 
by SVM interns, including cross-border corporate events as part of the so- 
cial venture expansion, local corporate relations and volunteering events, 
craft workshops and cultural fairs, as well as social media commercials. 
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Assessment Type 


Both formative and summative assessments are employed throughout the 
course. This combination of assessment methods captures student learning pro- 
gressively and prompts timely reflection of students when they are involved in 
out-of-classroom activities. 


Main Assessment Areas 


Problem-solving Skills: students are expected to be entrepreneurial and 
to help manage the routine problems or unexpected challenges of social 
ventures. With partner-oriented assessment through periodic and final 
feedback reports compiled by the general manager (A3), students receive 
suggestions on their problem-solving skills for their self-reflection and on- 
going improvement. The weekly reports (A2) then allow further space for 
students to write down their reflective analyses on the problem-solving 
process. Video making (A4) is also new to a lot of business/ social science 
students, testing their problem-solving skills. 
Communication Skills: To create a feasible working Action Plan (Al), 
students need to communicate effectively with the social ventures to de- 
velop a clear mutual understanding of the expectation on the scope of 
work. Upon receiving periodic feedback reports from the general manager 
(A3), students are expected to actively seek feedback and suggestions for 
improvement. Based on such feedback and follow-up conversations be- 
tween the manager and the course instructor, the course instructor then 
assesses the work performed by the student during the internship period. 
For the video assignment project (A4), students are advised to collaborate 
with their supervisor/manager concerning the nature, voice, or topic of 
their videos, so as to produce videos that will benefit the social ventures 
in some way. Proactive communication is thus assessed in multiple ways 
in this course. 
Collaboration Skills: in this course, one partnering social enterprise of- 
ten hosts several student interns in one semester to work in different ar- 
eas of business development (e.g. Fair Employment Agency hosted four 
students — a marketing intern, a community engagement intern, a digital 
marketing intern and a Sunday domestic worker outreach volunteer — over 
one semester, while Baby Hero hosted five students — a visual strategy and 
interactive design intern, a business development intern, an e-commerce 
strategy intern, a centric strategy & social media intern, and an inbound 
marketing strategy intern). 

Students who are placed at the same company therefore would work to- 
gether to deliver business strategies/activities of the social ventures and to 
produce videos (A4) for the final assessment of the course. Collaboration 
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skills are thus essential and clear evidence can be observed and shown 
through the (A3) and (A4) assessments. 

Service Leadership & Social Impacts Management: A lot of the students 
have signed up for the course as they are motivated by the possibility of 
finding a meaningful way to do real business and help the underprivi- 
leged. During the internship, students get the opportunities of establish- 
ing direct contacts with niche ideas for social impacts; general managers 
at the partner social ventures act as industrial mentors to cultivate the 
service leadership of students in finding new solutions for entrenched so- 
cial problems. This interactive process is assessed through weekly reports 
(A2), feedback reports (A3), and the reviewed action plan (Al). The video 
production and publishing on social media (A4) engage students with the 
wider public in disseminating key messages of the social ventures. 
Academic Knowledge: Students are expected to apply acquired knowl- 
edge in business management, accounting and marketing to operating 
real-world social ventures. This can be assessed by the general manager 
who has direct experience of how the student is handling practical busi- 
ness problems (A3). Students can also apply business concepts in analysing 
the problems they perceive or encounter during the internship, in the writ- 
ing of weekly reports (A2) and action plans (A1). 

Values for Reflection: Reflection on job performance is assessed in the 
weckly reports (A2). Reflection on personal growth and the work accom- 
plished in the course is one of the suggested topics of the video project 
(A4). The short write-up as one of the deliverables of the video assignment 
project (A4) is also a way to push the students to further reflect upon what 
they have produced with the video — i.e. their inspirations, the purpose of 
the video, and what they have learnt from the process of making the video. 
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Teacher’s Stories 


David Bishop 


Professional Engagements 


David Bishop holds a Juris Doctor from The Georgetown University Law 
Center, and a Bachelor of Arts in History and Mandarin Chinese from 
Brigham Young University. Besides teaching at HKU, David is an active 
social entrepreneur — he is the Founder of Soap Cycling and Social Impact 
Public Offering (Social IPO), and the Co-founder of Fair Employment 
Agency and Fair Employment Foundation. With legal practice qualifica- 
tion, he provides consultations to companies about ethics, corporate social 
responsibilities (CSR), pro bono, and social entrepreneurship initiatives. 
During his broad legal experience in the United States and China, David 
worked on major commercial mergers and acquisitions (M&A) deals, par- 
ticipated in negotiations, as well as acted as an outside counsel on various 
corporate matters for Multinational Corporations operating in Asia. 


Motivation 


The course coordinator of SVM, David Bishop, founded a number of 
the SVM partner social enterprises, namely Soap Cycling, Fair Employ- 
ment Agency and Social IPO. Soap Cycling became the foundation of the 
course SVM and provided most of the social venture internship opportu- 
nities when the course was first offered. 


°  Attheearly stage of the Soap Cycling operation, David stated that “About 
a year and a half ago, I was trying to find better job opportunities for my 
students, and to learn leadership and management, and do real things.” 

e The process of starting up a social business is being turned into a 
teaching instrument to engage student learning. *I would say (Soap 
Cycling is) efficient, impactful and empowering. We get a lot done 
with very little inputs, in terms of economic inputs — we don't really 
ask for funding generally speaking." 

° So we're really efficient in terms of what we can do. But we are 
also very impactful, because we can take that little amount of inputs 
going in, we can impact the lives in so many ways — we are able to 
do things for the environment, to help young people and empower 
them, and to work with NGO and to really boost their program 
and help them provide sanitation amenities to people who need it. 
The reason I started this (Soap Recycling) initially was really as 
a teaching tool for my students at HKU, and I wanted to give 
them an opportunity to learn how to do real stuffs." 

(Bishop, 2017) 
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Ripple effects of social impact have been created since the social venture 
was founded and the SVM course was launched. Mutual learning between 
students and the teacher further inspires the teaching process. 


e “The best thing I heard about Soap Cycling is that it has inspired other 
people to create businesses, to be entrepreneurial in ways that solve 
social issues. There have been a number of students that have left Soap 
Cycling and they have started their own social enterprises. We also have 
a whole bunch of other upcycling projects that have started in relation 
to Soap Cycling - recycling all kinds of things, sometimes fabrics, also 
soaps and soap bottles. So it has been great to see that people are taking 
the idea that we originally started with, and they are expanding it to 
ways that we wouldn’t have thought of at first.” (ibid) 


Challenges 


Teachers organising out-of-classroom experiential learning activities for 
students often bear enormous responsibilities. Effective responses with 
sensible solutions are critical when placement-related accidents happen. An 
accident on the factory floor of the soap recycling machinery took place 
when a student intern was on duty. As the course coordinator was travel- 
ling, some miscommunication about the incident occurred between the 
university management and the student. 


Students’ Side of Stories 


° = “T love working with the Baby Hero team, because of their enthusi- 
asm and dedication to the work and the cause, and I am learning a lot 
from them” (Dec 2016, Video Assignment). 

e “Interning with the expansion team has given me a different per- 
spective on the time and effort that goes on behind the scene of run- 
ning a social venture. Involving myself in both the Chow Tai Fok and 
Hilton events have [sic] taught me the values of planning ahead and 
being prepared. Overall this has been a great opportunity to learn 
new skills, make new friends, and most of all have a lot of fun” (Dec 
2016, Video Assignment, an exchange student from Australia). 

e “Its an opportunity to explore yourself, to get to know the part of 
you that you have never discovered. It is a change, and I love this 
change” (Dec 2016, Video Assignment). 

e “The course was challenging academically, but great fun! I have been 
given the opportunity to solve real-life problems, to manage a team, 
and gain hands-on business experience. The team consisted of several 
different nationalities, which have been interesting and rewarding - and 
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this has also improved my communication skills. I felt valued during 
the semester, and therefore strongly motivated. I highly, and genuinely, 
recommend considering this course — if you want to improve your 
business skills, improve your communication skills, and implement ac- 
ademic business concepts in a practical way; this is the right course for 
you!.” (an exchange student from Norway) 


Featured Video 


e SVMCourseYouTubechannel-collectionsofpastVideoAssignments: https:// 
www.youtube.com/channel/ UCOMO_tCDMgPb9MXBeJ KIYpA/videos 


References 


Bishop, D. (2017). BUSI2812 Social venture management internship course. The univer- 
sity of Hong Kong. Available at https://docs.wixstatic.com/ugd/841666_7cOff2e5af- 
b8443ca57830c9f28ce64b.pdf 

Course website: https://www.svmcourse.com/ 


8.9 Case 9: “Camping for Ornithology — Assessing 
Student Learning in a Field-Intensive Biology Class", 
Dalhousie University, Canada 


Field trips constitute the main component of coursework in the Ornithol- 
ogy course (BIOL 3622) at Dalhousie University in Canada. The course 
is open to undergraduate students in Biology after their second year, with 
a total of 15 students in each class. It runs for two weeks (from Mon- 
day to Sunday) in the summer as part of the university's SEASIDE (i.c. 
Summer Education & Applied Science Institute at Dalhousie in Ecology) 
programme that offers ecologically relevant training of practical scientific 
techniques. With condensed lectures, team exercises, hands-on labs and 
eight days in the field, students gain an overview of avian biology and 
techniques for the scientific study of bird populations, including identifi- 
cation by sight and sound. All the lectures and labs are planned in the first 
week to prepare students for the field trips. 

The field trips cover a one-day study in Halifax and a full-week camping 
at a field station in southwestern Nova Scotia to conduct research on birds 
in their natural habitats. This field-intensive class is hands-on and applied, 
where students gain practical experience with textbook knowledge. 
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Distinctive Features: 


e Featuring a wide diversity of practical learning activities through a 
combination of lectures, labs, a virtual field trip, and duo field visits 

e Engaging students comprehensively with three versions of field trips 
and team-based project work 

e Stressing on the values of learning by doing hands-on exercises 


Designed Learning Outcomes (LO) 


I Explain the behavioural, morphological, and physiological character- 

istics that distinguish the Class Aves from other animal taxa 

II Identify and understand general themes in avian ecology and evolu- 
tion (e.g. communication, mating, foraging, feathers, flight) 

III Appreciate the conservation issues affecting bird species, populations, 
and communities 

IV Identify most Nova Scotia Forest bird species (and others) by sight 
and sound 

V Generate relevant and detailed field notations of bird behaviour 

VI Integrate knowledge of principles and methods into the design and 
implementation of a research project that effectively addresses a re- 
search question about bird ecology 

VII Communicate scientific ideas effectively both in writing and through 
oral presentations 


Coursework Teaching & Learning Activities 


e  Pre-departure Lectures (4 sessions, 3 hours each, Day 1-4) & Lecture 
Test (Day 6) 

e  Pre-departure Labs (4 sessions, 4 hours each, Day 1-4) 

e Virtual Field Trip: Long Point Bird Observatory Migration Monitor- 
ing Bird Banding Station - live demonstration over a video conference 
call with an active banding operation (1 hour, Day 3) 

e Field Trip 1: Dingle Park in Halifax — practicing field skills with note- 
book entries (Day 5) 

e Field Trip 2: full-week camping at a field station in southwestern Nova 
Scotia (Day 7-13) 
e Surveys of bird (e.g. owls) habitats 
e Practicing field methods; 
e Team field projects (Day 9-12) 
e Field ID Quiz (Day 13) 

° ` Posttrip Summary: 
e Data analysis & report preparation (Day 14) 
° Symposium of class presentations (Day 15) 

Source: Dalhousie University (2017) Course Syllabus. 
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Assessment Type 


A wide variety of formative and summative assessments are adopted in this 
course to enhance student experiences through engaged and practical learning. 
They carry relatively equal weighting. 


Main Assessment Areas 


Holistic Competencies: Designed with learning by doing, this field- 
intensive class focuses on developing students’ hands-on aptitudes and 
problem-solving skills. The course LO also specifies the importance of 
enhancing students’ communication skills in terms of interpreting com- 
plex scientific ideas in both verbal and written ways. These are assessed 
through written reports of lab exercises (A1) and field reports (A7), as well 
as symposium presentations (A4) and participation (A5). Constructive 
mutual learning is encouraged through the symposium participation (A5) 
assessment that gives credit to students for raising questions and stimulat- 
ing discussions. Students in the field are also engaged collaboratively, by 
forming a team of 3 to formulate research ideas and implement a research 
plan in the field team research project. The outcomes of collaboration are 
presented in a final symposium (A4). 

Knowledge Application: This field-intensive class is also delivered by 
research-informed teaching. Integrating knowledge of principles and 
methods into the design and implementation of a field research project 
is an important course LO. To equip students with sufficient discipline- 
specific knowledge, there are focused and condensed lectures for them 
prior to entering the field. 

Their knowledge acquired from lectures is then assessed in a Lecture Test 

(A2). This newly acquired knowledge is also practiced in lab sessions (A1) 
before actual field experience. At the end ofthe field trip, students! knowledge 
in avian biology is once again examined by the Field Quiz (A3). The students 
then need to come up with a research question to be addressed by applying 
their knowledge in the field. They carry out the field research project by inte- 
grating knowledge, methods and practices. The result of this research project 
is presented and reported at the end of the field trip (A4 & A7). 
Reflection: Students are required to prepare and compile their own Field 
Notebooks (A6). The notebooks should contain documentation of field ex- 
perience featured with focal observations of the unexpected, which facilitate 
self-reflection of students during and after their field trips. In the final report 
(A7), students are also asked to reflect upon if they have any recommendations 
for future studies based on their field team research project. The final sympo- 
sium provides a platform for sharing and reflection, with project presentation 
(A4) and assessment of student participation (A5). Students are encouraged 
to ask constructive questions to their peers to stimulate reflective discussion. 
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Teacher's Stories 


Dr. Cynthia A. Staicer 
Dr. Sarah E. Gutowsky 


Professional Engagements 


Cynthia Staicer holds a Master of Science from Northern Arizona Univer- 
sity, and a PhD from the University of Massachusetts, Amherst. She is the 
faculty member in charge of the SEASIDE summer programme. 

Sarah Gutowsky holds a PhD in biology from Dalhousie University. She 
teaches part-time at the university while providing freelance consulting, 
and expedition guiding services. 


Motivation 


This course is delivered with research-informed teaching. Its design is 
driven by the research interests of Staicer (2018) in the development of 
ecological monitoring programme, the effects of forestry and recreation 
on bird populations, and the use of vocal behaviour of male birds to iden- 
tify their breeding status. 


Students? Side of Stories 


e “Ornithology is the best class I have taken thus far. Your interest in 
birds increases at least ten folds. Dr. Staicer really makes you have 
an understanding and respect for the winged beauties. When you go 
outside after the class you know what species are singing and calling. 
Walking in the woods is not the same anymore!.” (Anonymous stu- 
dent rating, 2008) 

e The course is “focused, intense, and exciting. Exactly what was 
needed to learn the abundance of information presented in the class." 
(Student A, official student rating 2014-2015) 

e “This particular course was almost entirely outside my discipline as 
a Marine Biologist. As such I was a little nervous to be going into a 
terrestrial field setting where I have less general knowledge. However, 
Sarah soon made it clear that that would not be an issue. The way 
in which she showed genuine interest in the students’ understand- 
ing of the practical material was excellent. She was knowledgeable, 
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approachable, and gave a definite sense of direction to all field activi- 
ties but without sucking the fun out of them. She also fostered curi- 
osity and welcomed students to share and discuss their opinions. ... 
The field portion of this class was incredibly well organized, I was 
never once stressed due to lack of direction or preparation. I had so 
much fun and learned so much, I really can’t stress enough how much 
I enjoyed this class. It was in large part due to Sarah, and I would 
jump at the chance to work with her again.” (Student B, official stu- 
dent rating 2014-2015) 

e “Sarah was one of the best professors I have ever had. She genuinely 
cared about every student in her class and focused every ounce of her 
energy to our learning and understanding of the material. The first 
day of class all the students were asked what they knew about birds 
and almost everyone said they knew nothing. It is now the last day of 
the course, and every member of the class has increased their interest 
and enthusiasm about birds drastically. There are many courses in 
university where there is no connection between the professor and 
their students, but this course was the complete opposite. I can ab- 
solutely say that I gained memories, experiences and knowledge that 
I will never forget!.” (Student C, official student rating 2014-2015) 

e <“... For me, she is the epitome of a role model, in terms of a female 
biologist. The field trip made us feel like a tight knit group. It was so 
much fun.” (Student D, official student rating 2014-2015) 


Featured Videos/ Photos 


e A slideshow of photos from the field is available on https://sarahgutowsky. 
weebly.com/teaching.html 
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8.10 Case 10: “Understanding Interconnections between 
Water Resources and Water Engineering — Assessing 
Overseas Engineering Field Trip,” University of 
Washington, the United States of America 


“Engineering Jordan: Water in an Arid Land” (CEE 497/ CEE 598/ ESRM 
490) is a credit-bearing elective course co-sponsored by the Department of 
Civil and Environmental Engineering and the School of Environmental and 
Forest Sciences at the University of Washington (UW), in partnership with 
the Jordan University for Science and Technology (JUST). UW students in 
engineering, science, or in environmental science and resource management 
can learn about the connections between water resources and water engi- 
neering in one of the most water-scarce nations in the world, Jordan. They 
also get to interact with Jordanian students, scientists, and engineers. There 
are around 8 UW students and 8 JUST students in the course. 

Primary learning is through field visits to water resources, drinking water 
treatment plants, and wastewater treatment plants located throughout North- 
ern Jordan. This learning is supplemented by seminars delivered by guest 
speakers representing local water professionals, government, and academics. 


Distinctive Features 


e Exploring the unique ecological landscapes and archaeological sites 
in Jordan 

e Fostering cross-country knowledge-exchange partnership with local 
stakeholders in Jordan 

e Interacting with local students and academics 


Designed Learning Outcomes (LO) 


I Provide students with the contextual background to apply critical 
thinking skills to water engineering problems on a regional scale and 
in an international context 

II Discuss interconnections among water source, water treatment, and 
wastewater reclamation using a specific example in Jordan 

III Describe water sources in Jordan and the demands placed upon them 
from Ist-hand observations 

VI List the steps in water treatment and wastewater treatment, and de- 
scribe the function of each step 


Assessment Cases around the World 291 


V Diagram an ancient water engineering system and use modern en- 
gineering theories to postulate how the ancient system may have 
worked 

VI Identify water resources and treatment plant operations that are af- 
fected by high ambient temperatures, by application of core engi- 
neering principles 

VII Identify cultural traditions and practices that impact water engi- 
neering in Jordan 

VIII Work in cross-cultural teams to achieve scientific outcomes 

IX Define *Cross-cultural Competency" within a science and engi- 
neering collaborative working setting based on personal experience 

X | Communicate science or engineering ideas to a targeted audience 
within a cross-cultural setting 

XI Prepare a compendium of notes and knowledge in a single resource 
(i.e. notebook) that can serve as a reference for future course work 
and professional practice 


Coursework Teaching & Learning Activities 


e  Pre-departure Lectures (1 hour per week; 10 weeks): theoretical and 
logistic introduction; culture, language and history of Jordan; quiz; 

e  Pre-departure Technical Field Trip around water treatment facilities 
across Washington State, the United States (minimum 2 hours) 

° Summer field trip in Jordan (4 weeks) to solve local water engineering 
problems 


1 Campus orientation & tour at JUST 
2  Lecturers delivered by invited Jordanian academics & practising 
engineers 
Technical tours 
Cultural tours 
Group activity on water governance 
Irbid city tour (optional) 
Overnight stays around Dead Sea & Wadi Rum/ Petra 
Hiking 
Project meetings with instructors 
10 Quizzes 
11 Student Project Presentations 
Source: UW (2018) Program Brochure. 
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Assessment Type 


A wide variety of assessment approaches are developed in this course, combining 
both formative and summative assessments. It is unusual that the formative ones 
make up the largest part of the course assessment and carry the highest weighting. 


Main Assessment Areas 


Holistic Competencies: Applying critical thinking skills in a new problem 
context and cultural setting is the core learning objective designed for 
this course. All of the assessment approaches (A1 — A6) are targeting this 
learning area. Another set of focus in the development of holistic compe- 
tencies in this course is on communication skills. With active classroom 
participation (AL), group discussions (A3) and group project presentations 
(A6), students are expected to communicate their science or engineer- 
ing ideas verbally and proactively. Team-based problem-solving skills are 
also stressed. The problem sets (A4) are designed specifically for training 
the problem-solving aptitude on site. The fourth holistic competency be- 
ing highlighted is creativity. The assessment of the group projects (A6) 
takes into account the creativity of student teams in terms of the appeal of 
graphical and visual presentation of scientific subjects. 

Cross-cultural Teamwork: Students get to enhance their collaboration skills 
in group discussions (A3), team-based problem-solving activities (A4), pro- 
ject meetings, and group projects (A6). In particular, through interacting 
with Jordanian students, WU students experience cross-cultural teamwork 
and develop “cross-cultural competency within a science and engineering 
collaborative working setting based on personal experience” (WU 2018). 
Knowledge Application: This course aims to help students apply engi- 
neering principles and equations in the analysis of water resources and use 
modern engineering theories to postulate the work of ancient water units 
in Jordan. The scientific content is the assessment focus of group projects 
(A6) where students are expected to clearly explain or demonstrate evi- 
dence that the team has studied and understands the topic focus. 
Reflection: The Study Log (A2) records the thoughts that students come 
up with as the course field trip proceeds. This facilitates students’ reflective 
journeys along their physical footsteps. The reflective thoughts of students 
build up the basis for group discussions in class (A3). Reflection then be- 
comes a recurring learning process. 
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Assessment Standards/ Sample Rubrics 


A2. Study Log Report/ 
Portfolio 
e 


Grade Descriptors 


Marks 


e Thoughtful notes with signs of critical thinking, observations, and 4 
questions focused on the course goals. Evidence of activities to 
answer one’s own questions. Preparation for discussions is evident. 
Shows improvement based on previous instructor comments. 

Some questions for discussions lack in-depth thought or 2 
independent thinking. Shows improvement based on previous 


instructor comments. 
° Some notes, no clear evidence of discussion preparation. 1 


Source: UW (2013) Course Syllabus. 


Areas Marking Criteria Marks 
Scientific * Scientific elements are clearly explained or 50% 
Content demonstrated. There is strong evidence that the team 


Presentation ° 


Creativity ° 


A6. Project 


Bonus ° 


has studied and understands the topic focus. 


The presentation is neat, organized, and 30% 
well-rehearsed. 


The presentation is graphically or visually interesting. 20% 
Engaged audience through the: use of verbal cues 

(tone, pitch and pace of speech); use of non-verbal 

cues to highlight presentation (body language, facial 
expression, hand gestures, and eye contact); 

Provided sufficient information within the timeframe 

in a logical and structured manner 

Materials from field trips are incorporated into the Up to 
presentation in meaningful ways (such as photos), or 5% 
other considerations at the instructor’s discretion. 


Source: UW (2013) Course Syllabus. 


Featured Videos 


e Photo gallery of the 2013 field trip http://courses.washington.edu/ce- 
jordan/photos.html 

e Student photographs of 2014 field trip http://www.anxhuynh.com/jordan/ 
c96jmcauwdy9p20ww3lqokhxfajh5c 

e Student blog post of the field trip in 2012 with photos and a video http:// 
seattletojordan. blogspot.com 
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8.11 Case 11: “Ready for the Chemical Industry? — 
Assessing Chemistry Internship Experience”, 
National University of Singapore, Singapore 


Final Year Internship in Chemistry (CM4299) is an elective of 16 credits 
for final year undergraduate students majoring in chemistry at the Na- 
tional University of Singapore (NUS), in which students are required 
to work full time in a chemistry-related industry for six months (a min- 
imum duration of 22 weeks). The detailed scientific merit of the actual 
work performed is not directly assessed. This credit-bearing internship 
is letter-graded, i.e. not merely pass or fail. To get a passing grade, the 
intern must prove that they have achieved the intended learning outcomes 
through proper maintenance of a reflective journal during the internship. 
The student while doing the internship is not allowed to undertake any 
other modules even during weekends or at night. 
This course only accommodates local and self-sourced internships. 


Distinctive Features 


e Giving initiatives to the students in sourcing the internship to experi- 
ence job searching process 

e  Stressing on quality assurance of the provision of approved internship 
by dispatching staff visits at the workplace, collecting student feedback 
and work-progress logs, and compiling on-site evaluation reports 

e Examining internship performance cautiously by balancing the work- 
place supervisor's view with the holistic view of selected academic ex- 
aminers on student learning 


Assessment Cases around the World 297 


Designed Learning Outcomes (LO) 


I [Communication] Demonstrate a satisfactory ability to communicate 
rationally and logically, concisely, clearly and effectively the nature of 
the internship and the intern’s learning outcomes 

II [Overall performance in the workplace] Discover and adopt the 
desired work attitudes, ethics, safety culture and professionalism 
through working interactions with supervisors, co-workers, clients 
and other people related to the organisations and institutions where 
the internship is conducted 

III Integrate knowledge and skills learned at the NUS into a workplace 
professional setting 

IV Reflect on the skills learnt and their impacts on the intern's personal 
and career development 

V Reflect and improve upon the intern's performance in the workplace 


Coursework Teaching & Learning Activities 


Pre-internship Procedures 


e Annual module briefing (late Jan/ early Feb) 

e Students seeking internship opportunities from companies 

e Internship offers 

e Internship approvals by the course coordinating committee 

e Students accepting the approved internship offers & informing mod- 
ule administrator about such decision 

e Assigning a Staff Advisor to each student intern by the coordinating 
committee 

e Pre-internship Briefing 

e Signing the Internship Learning Contract 


Internship 


e Quality assurance: Staff Advisor's visit at workplace, meeting with the 
intern supervisor, and filling out an On-site Evaluation Report (Week 
6; compulsory) 
e Staff Advisor's follow-up meeting (if necessary) 
° Performance examination & Internship Quality Assurance Report by 
two examiners chosen by the committee and the Staff 
Source: NUS (2018) CM4299 Guidelines. 
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Assessment Type 


Most of the graded assessments in this course are of summative nature. The form- 
ative assessment is either not directly graded (i.e. the Reflective Journal) or carries 
little weightage (A1). Performance of the student intern is assessed in an all-rounded 
way by the coordinating committee, the selected examiners, the Staff Advisor, and 
the workplace supervisor. Apart from assessing students, the assessments also serve 
as quality assurance instruments for course delivery and internship providers. 


Main Assessment Areas 


Holistic Competencies: The pre-internship briefing session makes students 
aware of the “additional skills they have picked up, or enhanced, during their 
tenure at the NUS” (NUS, 2018). The intended learning outcomes have iden- 
tified the importance of generic skills in personal and professional develop- 
ments (LO III & IV). Holistic competencies are assessed by the internship 
supervisors in the workplace performance assessment form (A2). Essential 
assessment items include the ability to learn, initiative taking and creativity, 
reliability, communication skills, critical thinking skills, leadership abilities, as 
well as interpersonal and teamwork skills. In the internship report (A4), stu- 
dents are advised to give attention to how skills learnt at the NUS are applied 
in the workplace. Students are expected to show or highlight how they trans- 
late a number of skills into the workplace. In the preparation of the portfolio 
(A5), students are assessed in terms of how their acquired skills are translated 
into the workplace, as well as their reflections on skill learning. 

Knowledge Integration: The integration of knowledge in professional set- 
tings is identified as one of the intended learning outcomes (LO III). This 
is assessed in the internship report (A4). Employment of technical skills as 
part of the knowledge of a chemistry degree is assessed in the workplace 
supervisor assessment (A2). 

Reflection: The course learning outcomes have attached significance to 
reflection (LO IV. & V.). Reflective practices are integrated into the course 
assessment, through the maintenance of a reflective journal by individual 
student interns and the writing of a reflective essay in the final portfolio 
(A5). The reflective component is purposefully designed in the course and 
is justified by pedagogical studies. The course design is informed by re- 
search findings on how reflective writing can sharpen students’ learning 
experience (Biggs & Tang 2011, p. 262). 


Assessment Standards/Sample Rubrics 


Each assessment approach (A1-6) in this internship course has its own rubric(s) 
and assessment criteria. This case study selects two sets of rubrics as a sample for 
the readers’ reference. The selection is made given the quality and robustness 
of rubric design, as well as the assessment weighting in overall grading. The 
rubrics for performance assessment by workplace supervisors (A2) and the port- 
folio (A5) are thus chosen. 
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Overall Performance: Evaluation of Student Intern 


Outstanding Very Good Satisfactory Marginal Unsatisfactory 
= D D D im] im] 
A [fa position were available within your company/ Yes No 
° organisation, would you recommend this student for 

employment? 

Comment: 


Are there any other additional comments you wish to add regarding the 
student intern? 


Technical Notes: In the actual form of A2[1], there is a box for comments 
underneath each row of evaluation dimensions as well as an extra column for 
scores to the right of the performance rating table. A2[1] contains responses 
to 14 evaluation dimensions and carries a 75% weighting in this assessment. 
Each evaluation dimension is weighted equally and scores maximally 6 marks, 
so the maximum mark would be 84. The total score for A2[1] then needs to be 
adjusted, being divided by 84 and multiplied by 75. For A2[2], it weighs 25% of 
A2. The answer to whether or not the intern would be recommended for em- 
ployment is used to moderate the overall performance with a matrix as follows: 


Outstanding Very Good Satisfactory Marginal Unsatisfactory 


Recommended 25 22 19 16 13 
Not recommended 21 18 15 10 5 


Evaluation Performance Rating 


Dimensions 
Excellent Good (B+,B,B-) Marginal (C+,C,D+) Unsatisfactory 
(A+,A,A-) (D, Si 0) 
Quality of As good, and Clear, readable, Frequent problems Major problems 
writing (LO is written in an prose. Good use with mechanics of with mechanics 
I) engaging manner. of transitions; language; occasional of language; 
Interesting to no problems awkward sentences awkward sentence 
Ke, read. with spelling, and poor transitions; construction; poor 
£ punctuation, reduced readability. or absent transitions; 
S or grammar. Minor problems frequently difficult 
£ Clear and logical of organisation or to understand. 
iG presentation; good logic; needs work on Weak organisation; 
< development of creating transitions sentences rambling; 
an argument; between ideas. ideas are repeated. 


transitions are 
made clearly and 
smoothly. 
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NUS Skills As good, and Shows/highlights | Shows/highlights Nonsensical 
translated recognises several what skills (more one skill learnt relationship drawn 
into wide ranging than one) learnt at NUS that was between skills 
workplace skills that were at NUS were translated into the learnt at NUS and 
(LO III) translated into translated into the ^ workplace. Unable translated into the 
the workplace. workplace. to properly recognise workplace, or no 
Additionally, skill sets picked up at evidence that any 
the skills were NUS, or enhanced skills were even 
generalised to at NUS which recognised at all 
be applicable not were utilised in the ` which were drawn in 
just to the tasks workplace. the workplace. 
undertaken in the 
internship. 
Reflection As good, and The essay presents Able to identify new The essay either 
on skills evidence of a good appreciation skills learnt, but are does not address any 
learnt internalisation of the new skills not well appreciated new skills learnt, 
(LO IV) of what was learnt and how and unable to see any or if it does, there 
learnt into a they impact significant impacts is no meaningful 
personalised personal and on personal and indication of 
model of good career development career development. how these skills 
practice. Good and is backed up Limited reflection impact the intern’s 
evidence of with relevant and ^ appears to have been personal and career 
reflection on own accurate support. performed. Weak development. No 
o performance. or poor justification evidence of serious 
= Generalises with portfolio items. reflection. Weak 
Ka conclusions and or no evidence for 
ë what was learnt claims made. 
m in new and 
= unfamiliar real- 
life contexts. 
Reflect & As good, and Able to use Able to use available Able to use available 
Improve able to further available information to information to 
(LOV) generalise self- information to self-evaluate and self-evaluate and 


evaluation beyond 
the internship. 
Suggests ways 

of improving 
professional 
performance 
generally. 


Source: NUS (2018) CM4299 Guidelines. 


self-evaluate and 
identify the full 
range of own 
strengths and 
weaknesses. Self- 
evaluation properly 
targets an area 

of improvement 
which was then 
attempted to 

be approved 

upon. Areas of 
improvement were 
specific to the 
internship. All 
observations and 
conclusions are 
properly supported 
by items in the 
portfolio. 


identify a few aspects 
of strengths and 
weaknesses. Little 
evidence of making 
any serious attempt 
to improve on the 
identified areas. 

Few ideas on how 
improvements could 
be made. Some 
evidence that the 
observations are 
backed up with items 
from the portfolio. 


identify limited 
aspects of strengths 
and weaknesses. 
No evidence of 
suggestions of 
ways to improve 
performance. Little 
if any evidence is 
used to back up 
claims from items in 
the portfolio. 
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8.12 Case 12: Understanding Contexts of Law and Legal 
Practice - The Undergraduate Legal Internship 
Programme, University of Wollongong, Australia 


The undergraduate Legal Internship Program (LLB397) offered by the 
School of Law at the University of Wollongong is a compulsory two- 
credit subject in which students have to complete a 20-day internship un- 
der the supervision of a legal professional. The objective of the program 
is to provide students with an understanding of the human, social and 
policy contexts of law and legal practice. By integrating the professional 
experience into the learning process, students can develop an under- 
standing of law in action through observing and perceiving the relevance 
and application of theory to practice during their internship. Students 
are expected to reflect upon the values, ethical standards and conduct of 
the legal profession and develop their own attitudes towards professional 
responsibility. 


Distinctive Features 


e Requiring all students to complete a legal internship as part of their 
law degree 

e Application for internship can be arranged by the School of Law or by 
students themselves, with approval from the School of Law 


Expected Learning Outcomes (LO) 


I Appreciate some ofthe practical aspects and social dimensions oflegal 
problems; 
II Relate the application of different areas of legal principles to the appli- 
cation of the different skills of research, communication, and practice 
that different types of legal work require. 


Pre-Internship 


During Internship 


Coursework Teaching & Learning Activities 


e Matching exercise between the Law School, possible internship pro- 
viders and student applicants. 


e Internship experience and journal entries (4 weeks/20 days). 
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Assessment Approaches 
Name Learning Activities Weight Aligned Type Generic Skills 
LO 
Al Identification On the first day of the Il. Formative Communication; 
of learning internship, students are collaboration; 
objectives expected to discuss with self-management 


their work supervisor 
and to identify a list of 
learning objectives and 
work activities for the 
internship period. This 
statement of learning 
objectives will be signed 
by the work supervisor 
and submitted to the 
Law School for approval 
within the first three 
days of the internship 


experience. 
A2 Reflective Students have to Ike Formative & Critical thinking; 
Journal prepare a minimum of Il. Summative problem solving; 


four journal entries on 
work and observations 
undertaken during the 
internship experience. 
Students also have 

to include a piece of 
reflection on how the 
internship has changed 
their view and/or 
understanding of 

the law. 


self-management 


(Continued) 
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Name Learning Activities Weight Aligned Type Generic Skills 
LO 
A3 Student Students have to submit - JL WWE Summative Problem solving; 
internship an evaluation form to self-management 
evaluation indicate whether they 
have met the learning 
objectives identified at 
the beginning of the 
internship. 
A4 Signed record Students are required E I. Formative & Self-management 
of attendance to maintain a record of Summative 


attendance during the 
internship experience, 
recording any 
variations to the agreed 
attendance schedule for 
insurance purposes. 


Assessment Type 


Both formative and summative assessments are employed in the module. Such 
a combination of methods allows assessment of students’ learning progress and 
outcomes after the internship, prompting students to record and analyse obser- 
vations and impressions during the internship. 


Main Assessment Areas 


Values for Reflection: Reflection on performance is assessed in journal 
entries (A2). Reflection on personal growth and learning outcomes at the 
end of the internship is assessed in the student internship evaluation (A3). 
The assessments encourage students to reflect on their actions during and 
after the internship, whether they have achieved their objectives and what 
they have learnt in the program. 

Self-management Skills: Students are expected to manage themselves 
throughout the four weeks of internship, setting objectives (A1) and mon- 
itoring their own learning progress through completing journal entries 
(A2) and evaluating themselves at the end of the program (A3). Being 
required to maintain an attendance record (A4), students are pushed to 
take responsibility for their own actions. The assessments test students’ 
self-management skills as students have to take initiative in the module, 
from applying for an internship to submitting the internship report. 
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Knowledge Application: Aiming to expose students to the law in opera- 
tion and different areas of legal practice, the programme expects students 
to apply and relate their internship experience to theories and knowledge 
learnt in lessons. By recording their experience in journal entries (A2) and 
reflecting on the experience, students further develop and reinforce skills 
and understanding of legal problems. Students may also analyse their suc- 
cesses and failures during the internship in the student internship evalua- 
tion (A3), reflecting on their workplace competencies and what they have 
learnt about demands and responsibilities in the legal workplace. 


Assessment Standards/Sample Rubrics 


A2. Reflective 


Journal 


The journal is marked based on the following criteria: 


a The amount of effort put in by the student; 

b The extent to which the student is able to explore and discuss issues 
encountered; 

c The quality of writing (e.g. clarity, creativity, diversity of concepts); 

d Demonstration of learning from the experience in terms of insights and 
conceptual depth; 

e Ability to critically analyse the experience; 


Ability to relate theories learnt at the university to their experience and 
Observations in the workplace. 


A3. Student 


Evaluation 


The evaluation form requires students to: 

a Attach a copy of objectives developed at the start of the internship and 
comment specifically on whether the student's experience achieved the aims 
identified, and say why/why not; 

b State any other general comments about the internship experience. 


Internship 


Report 


All components of the internship report (including reflective journal, student 
evaluation and attendance record) must be submitted within two weeks 

of completing the internship. The Internship Report will be graded as 
Satisfactory or Unsatisfactory. The supervisor's evaluation is optional, and if 
provided, the evaluation does not form part of the assessment. 


Teacher's Stories 


John Littrich, Senior Lecturer, School of Law, the University of Wollon- 
gong (UOW) 


Professional Engagements 


John Littrich was admitted as a solicitor to the Supreme Court of NSW 
in 1989 and the High Court of Australia in 1990. He practised between 
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1989 and 2002 in the Illawarra and Shoalhaven regions primarily in the 
litigation field as well as a year with the NSW Coal Association from 1994 
to1995 as an Industrial Relations Advisor. He became an NSW Law So- 
ciety Accredited Specialist in Family Law in 1999, then began teaching at 
UOW in 2002. He was appointed as a Lecturer in 2004 and since then has 
coordinated a range of subjects in the undergraduate LLB program, also 
undertaking the post of Litigation Co-ordinator of the UOW Graduate 
Diploma in Legal Practice course. In 2014, John Littrich was appointed 
to the position of Discipline Leader, Clinical Legal Experience and Profes- 
sional Engagement, overseeing the Law School’s unique Legal Internship 
Programme and other experiential learning programmes. His co-authored 
text with Professor Ainslie Lamb, Lawyers in Australia, has been used to 
teach legal ethics at UOW and other universities for a number of years. 


Motivation 


In 2008, John Littrich and M.T. O’Brien issued a journal, which wrote 
“Students learn most effectively (or deeply) when they are called on to par- 
ticipate in activities that engage them in role-sensitive, experiential or con- 
textualised instructional activities that require them to mobilise learning 
into action" (p. 62). As the academic coordinator of the Legal Internship 
Program, Littrich believes in the importance of experiential learning and 
the importance of practicing various skills in the professional field. This 
echoes the aims and objectives of the Legal Internship Program, which 
focuses on exposure to the *practical dimensions of legal principles? and 
“aspects of law which cannot be learned from reading or hearing about it” 
(UOW 2018, p. 4). The main motivation behind the Legal Internship Pro- 
gram is to provide students with the opportunity to observe law practices 
and assist them in their future careers. Skills are acquired and reinforced 
in the experience of professional practice. 


Challenges 


° As completing an internship is a requirement for the law degree at UOW, 
it is challenging to build up and maintain contacts with a large number 
of partner organisations to provide internships. It also takes adminis- 
trative effort to match students with appropriate internship placements. 


References 
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March 21, 2018, from https://lha.uow.edu.au/content/groups/public/@web/@lha/@ 
law/documents/doc/uow209065.pdf 


Conclusions 


This chapter provides a large database of empirical cases from around the world. 
As some of the information including videos cannot be shown in text format, I 
have placed a link here (https://ar.cetl.hku.hk/EL) for easy access. There are 
also more cases on the website and since only a limited number of cases can be 
included in the book, I invite you to visit the website for more information. 


Questions to Ponder 


e Most experiential learning courses employ a combination of assess- 
ment approaches, but at the same time, more assessment means dis- 
traction from meaningful activities, what should I do? 

e As mentioned in the book, experiential learning requires support 
from different levels (particularly the senior management level). The 
cases shown in this chapter seem to be well-funded and supported. 
How can I, a teacher with limited funding and support, initiate and 
implement my own experiential learning courses successfully? 

e Jt looks like feedback is still an after-thought, as most experiential 
learning cases do not appear to prioritise the use of feedback. Should 
I spend time integrating feedback into the design of an experiential 
learning programme? Or should I spend more time on the activities 
so students will be more engaged? 


Personal Reflection 


I want to thank all the teachers who have done wonders in their experien- 
tial learning courses and projects. I have first-hand experience on how dif- 
ficult it is to build a successful experiential learning course. It takes time, 
resources, support, patience and luck. I also thank you for the outstanding 
work you have done on your assessment. It is through these cases, we 
teachers, researchers and in turn, students can develop better. 

Getting support is always one of the most difficult hurdles for teachers. 
Ihave talked to many teachers, and most of them started small. It is always 
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easier to get support and resources when you have some successful cases 
to showcase. 

If anyone has more excellent cases to showcase, please contact me. It will 
be my honour to showcase your work on our website. 
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9 Assessing Experiential 
Learning with Technology 


Technology comes and goes, it does not drive learning. Pedagogy should always 
come before technology. 
— Chan, CKY 


Introduction 


Due to the globalisation and digitalisation of education, technology is increas- 
ingly used in various domains of the field, including experiential learning. In 
recent years, the outbreak of the coronavirus pandemic in 2019 has also unques- 
tionably catalysed the use of technology in experiential learning. Studies have 
shown that the younger generation prefers working on experience-based projects, 
enjoys opportunities for collaboration, and expects more interactive, engaging 
activities that integrate the use of current technologies (Vito, 2013). Technol- 
ogy is widely perceived as an effective tool for experiential learning as it encour- 
ages innovation (Murphrey, 2010), supports different learning styles (Beckem & 
Watkins, 2012), increases students’ self-efficacy (Douglas-Lenders et al., 2017), 
allows students and teachers to communicate outside the classroom settings and 
promotes the development of holistic competencies (Hu, 2015). 


9.1 Role of Current Technologies in Experiential 
Learning 


Technology can be integrated into experiential learning as the main tool for 
learning, or as a supporting element in the teaching and learning process. 
When technology is used as the main tool, it is fully integrated into the in- 
structional process with learning activities being planned and implemented 
around it, such as in simulation-based and digital game-based learning. As 
a supporting tool, technology is used to supplement learning by providing a 
means of developing and storing digital resources for self-directed learning, 
documenting and assessing learning experiences, as well as synchronous and 
asynchronous collaboration. 


DOI: 10.4324/9781003018391-9 


Experiential Learning with Technology 317 
9.1.1 Technology as the Main Tool for Learning 


As the main mechanism for experiential instruction in simulation-based learn- 
ing, current technologies are commonly used in medical and healthcare fields 
such as dental education (Lemaster et al., 2016), nursing education (Koivisto 
et al., 2017; Weideman & Culleiton, 2014), and counsellor education (Lowell & 
Alshammari, 2019) to facilitate the development of students’ knowledge, com- 
petencies, and attributes in a safe environment without posing potential risks 
to patients and clients. Weideman and Culleiton (2014) show that by using a 
virtual pregnancy model, nursing students could monitor a character’s virtual 
pregnancy as it progressed over a specified time period. The virtual pregnancy 
model had embedded questions and feedback to guide students throughout the 
process of providing health education to the virtual character. In Lowell and AI- 
shammari's (2019) study, counsellor education students applied the interviewing 
and diagnosis skills they had learnt in an online 3D role play with patient avatars. 
These activities provide opportunities for experiential training of professionals 
in a low-risk environment where errors in diagnosis and clinical decisions do not 
have any detrimental impact on human subjects but serve as teaching points for 
future improvements. 

Technology-enhanced simulations are also used in other fields such as supply 
chain management and the science, technology, engineering, and mathematics 
(STEM) disciplines to allow students to immerse themselves in authentic work 
processes in a virtual environment that parallels the real-world context. For ex- 
ample, business students adopt different roles in logistics, sales, purchasing, and 
accounting to participate in a virtual supply chain management process that in- 
cludes sales and operations planning, inventory control, stock transfer planning, 
and periodic inventory review (Angolia & Pagliari, 2018). Similarly, engineering 
students can have hands-on experience of the effect of bank angle and the speed 
based on the radius of turn as they manoeuvre a virtual aircraft using a flight 
simulator (Aji & Khan, 2015). With the use of current technologies, it is possi- 
ble to provide students with such practical experiences which are otherwise too 
costly and unfeasible to be carried out in person. 

Simulations and digital games are closely related and sometimes used inter- 
changeably in the experiential learning literature. In some studies, serious games 
also known as gamification are integrated into simulation-based learning to provide 
a motivating way to practise concepts learnt in class (Le et al., 2015, 2016). For 
example, in Le et al. (2015) study of technology-enhanced construction safety edu- 
cation, the students performed construction tasks using motional avatars to identify 
and correct potential risks in a simulation game. The virtual scenarios were designed 
to closely resemble real-life construction sites to help the students gain practical 
on-site safety experiences in a safe environment. An advantage of simulation game- 
based learning is that it offers a stimulating and engaging approach for encouraging 
collaborative and experiential learning in a close-to-reality virtual environment (Le 
et al. 2015, 2016). 
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9.1.2 Technology as a Support Tool in the Learning Process 


In its role as a support tool for learning, technology sometimes acts as a digital 
repository for information and learning resources. Online learning manage- 
ment systems such as Moodle, Canvas, and Blackboard are learning platforms 
that store course content and learning materials, host discussion forums and 
blogs, and allow easy and accessible messaging by students anytime and any- 
where. In the flipped classroom approach, lecture content videos can be up- 
loaded to an online learning platform, so that students can watch the videos 
in advance to equip themselves with fundamental concepts before attending a 
face-to-face class or experiential learning activity (Peisachovich et al., 2016). 
In addition to videos, other types of content such as cases, exercises, data, and 
pictures can also be provided in a virtual classroom to stimulate online discus- 
sions (Le et al., 2015). 

Technology provides a means through which learning experiences and arte- 
facts are recorded and documented in a range of digital formats. At the Univer- 
sity of Hong Kong, law teachers teaching tort law are capturing pre-recorded 
videos at different locations to bring students out of their classrooms. Teachers 
may also ask students to step outside to their neighbourhoods and find torts of 
different types in their everyday lives. The use of videos and photos is becoming 
more popular, with cameras in mobile devices being readily available. Teachers 
need to provide clear criteria for the assessment of these new approaches, as video 
and photo editing, sizes and length of videos, and submission approaches are 
challenges that need to be considered. 

In the art, creative, and technology disciplines, students are encouraged to use 
editing software to compile audio files, images, motion graphics, and video clips 
for an e-portfolio of their internship experience (Mirrer, 2010). E-portfolios al- 
low students to document their learning process, share their experiences with 
a specialised learning community, and present their competencies to potential 
employers in a digital form (May et al., 2016). Also, the history of students’ 
communication and work processes that are documented on web tools or online 
learning platforms can be analysed to evaluate learning and programme effec- 
tiveness (Hu, 2015; Le et al., 2015; Mundkur & Ellickson, 2012). Students can 
also reflect on their own performance by reviewing the learning artefacts they 
have produced and preserved digitally (Mattera, Baena, & Ureña, 2014), this is 
elaborated in Section 5.6 on rework reflection. The use of current technologies 
such as videocasting tools, learning management systems, and virtual classroom 
software makes it possible for students to focus on key elements of their learning 
experience. 

Both synchronous and asynchronous web tools are useful for collaboration. 
Synchronous web tools, such as virtual platforms, support real-time discussions, 
editing of work, and presentations to enhance a sense of belonging to a learning 
community and prevent feelings of isolation (Mundkur & Ellickson, 2012). On 
the other hand, asynchronous web tools — which are not time-sensitive — allow 
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flexibility in communication and collaborative work at a time and place that is 
convenient to the users. Examples of such tools are email, online forums, blogs, 
and wikis. Wikis are particularly useful as mass authoring tools that enable stu- 
dents to work collaboratively in an online editorial process to support experi- 
ential learning (Pardue et al., 2013). In addition, digital media products such 
as movies or video clips that students have created for an experiential learning 
project can be published online on platforms such as Facebook (Meta), YouTube, 
and Flickr for their classmates to view and facilitate discussions asynchronously 
to encourage collaborative learning (McCormick et al., 2010). In experiential 
learning programmes that involve external stakeholders, off-campus internships, 
or geographically diverse participants, the use of communication technolo- 
gies supports online discussions and timely feedback sessions (Chilton, 2012; 
Lackéus & Middleton, 2018; Mettidinen, 2015; Zettinig et al., 2015) to ensure 
that the programme goals are achieved. 

Zhu et al. (2016) proposed a framework for designing and evaluating a 
storm drain mapping project that required the use of GPS devices in an 
undergraduate introduction to management information system course. 
Formative evaluation was done through verbal checking of students’ concept 
understanding, whereas summative evaluation was conducted through face- 
to-face interviews with students regarding their assessment of their project 
design and understanding of GPS before and after the project. In addition, 
the students completed a questionnaire consisting of scale items and open- 
ended questions on the perceived authenticity, challenges, competencies, and 
level of autonomy in the project. The students commented that the project 
had improved their understanding of GPS technology, helped them acquire 
relevant technical skills for using GPS devices to collect data, and increased 
their interest in information technology. In another study, Mendez-Carbajo 
and Davis-Kahl (2019) reported their comparison between assessment of tra- 
ditional scholarly capstone research projects and technology-enhanced ser- 
vice capstone research projects in an undergraduate economics programme. 
The traditional scholarly approach involved students searching library data- 
bases, assessing data sources, and producing research outputs in the forms of 
written papers and oral presentations. The service-learning approach, on the 
other hand, required students to work in partnership with local government 
agencies, share data on residential foreclosures via geographical information 
software (GIS), and analysed economic impacts using the Impact Analysis 
for Planning software. The research outputs were an independent research 
project and presentations at a regional scholarly conference. Based on the 
assessment of 85 research projects, the researchers found that those with a 
service-learning focus showed a stronger mastery of empirical methods and 
visuals than those with a traditional scholarly focus. Overall, the service re- 
search projects appeared to be, while only marginally, more competent than 
the scholarly research projects. 
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9.2 Effects of Current Technologies on Experiential 
Learning 


9.2.1 Positive Effects 


Various studies have shown that the use of current technologies has a positive 
effect on experiential learning. Murphrey (2010) investigated the effects of 
interactive technologies on student engagement in experiential learning activ- 
ities and concluded that technology encouraged students to be innovative in 
their work and to take ownership over their ideas (Murphrey, 2010). Besides, 
Mundkur and Ellickson’s (2012) study shows that technology-supported ex- 
periential learning provided opportunities for individual as well as collabora- 
tive learning and application of learning to real-world contexts. Based on the 
participants’ feedback, reading peers’ postings on the discussion boards en- 
couraged self-reflection as it inspired the participants to think about their own 
work and enriched their perspectives on the issues discussed. In Weideman 
and Culleiton’s (2014) simulation-based research using a virtual pregnancy 
model, the students were able to learn and apply appropriate obstetrical con- 
cepts in the virtual clinical scenarios they encountered. The students perceived 
that working with the virtual pregnancy model increased their competency 
in clinical judgement, clinical inquiry, caring practices, and as a facilitator of 
learning. 

The attractive colours, graphics, audio, and in-built tutorials in gamified 
learning tools add a fun factor to experiential learning (Murdoch, 2019), 
motivating and engaging students in the learning process (Catalano et al., 2014; 
Le et al., 2015; Le et al., 2016). In digital game-based learning, students tend to 
focus fully on the game, which is the learning task itself. Failure to complete the 
game can be considered as an opportunity for the facilitator to evaluate together 
with the students how their decisions have led to failure in the game, either dur- 
ing the game or post-game debriefings (Catalano et al., 2014). 

Apart from enhancing student engagement, virtual simulations support 
the development of holistic competencies such as presentation and commu- 
nication skills (Beckem & Watkins, 2012). In Douglas-Lenders et al. (2017) 
study, the research participants reported a higher level of work-related self- 
efficacy after undergoing simulation-based training in occupational health 
and safety. The use of virtual groups is becoming a standard practice in in- 
ternational corporations, and the ability to collaborate virtually is therefore 
an important soft skill of the 21st-century workforce (Leonard et al., 2011). 
In addition, the integration of information and communications technology 
(ICT) in experiential learning has been found to increase students’ willing- 
ness to take risks (Bandera et al., 2018) and encourage the development of 
self-regulated learning competencies, comprised of communication compe- 
tencies, computing competencies, analytical competencies, creativity, and 
team competencies (Mattera et al., 2014). Apart from holistic competencies, 
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research has also shown that current technologies have a positive impact 
on student attainment of subject-specific learning outcomes in experiential 
learning (Aji & Khan, 2015; Geng & Alani, 2018). 


9.2.2 Negative Effects 


The use of current technologies in experiential learning is not without its critics. 
Some writers have cautioned against the uncritical use of current technologies in 
outdoor experiential learning as they believed that current technologies, such as 
smartphones and tablets, have the potential of distracting students from totally 
immersing themselves in the natural surroundings, unfamiliar culture, and au- 
thentic interactions with the local community (Hills & Thomas, 2019; Shultis, 
2012; Smith et al., 2018). Interaction within the physical and socio-cultural 
contexts of a place is an important aspect of experiential learning that cannot be 
replaced with technology. 

In the study conducted by Smith et al. (2018), the participants in a study 
abroad course in New Zealand recounted their experience of being separated 
from technology as one that afforded them valuable quiet moments for genu- 
ine self-reflection while being totally immersed in the rural settings. To explore 
the concept of sustainable development in tourism, Schott and Marshall (2018) 
trialled a virtual field trip using a software-based virtual Pacific Island and vir- 
tual reality headsets involving six university staff members and five students. 
Although the participants reported a great sense of immersion in the virtual 
environment, they were deprived of opportunities to have active, meaningful 
interactions with the local community in the simulation. Such shortcomings 
may limit the potential of technology to enhance experiential learning outcomes. 
Bursztyn et al. (2017) compared the learning of geoscience concepts between 
students who were taught in a traditional curriculum and students who partic- 
ipated in augmented reality field trips using a pre-test/post-test design. The re- 
sults show that there was no significant difference in the learning gains attained 
by both groups. The findings from these studies inevitably cast doubt on the 
extent to which current technology use positively impacts experiential learning 
outcomes. 

Hills and Thomas (2019) argue that the decision to use or not use digital 
technologies in outdoor experiential learning should be carefully examined from 
three aspects: pedagogical considerations, affordances of digital technology, and 
consequences of decisions. In a similar vein, Wang (2011) warns against integrat- 
ing virtual technology in higher education programmes without a sound peda- 
gogical basis. He questioned the transferability of experiential learning from the 
virtual world to the real world due to the sometimes unrealistic virtual settings 
and the often-predictable causality of interactivities in virtual simulations, which 
differs greatly from the complex cause and effect relationship in natural human 
behaviour. 
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9.3 Case Study Using Technology 


With industrial revolution 4.0, there is an increasing number of technology- 
enhanced learning tools. When selecting a suitable technology tool to use in 
supporting and assessing experiential learning, teachers should consider the jus- 
tifications of their integration into teaching and learning and should always place 
pedagogy first before technology. In the following case study, I will demon- 
strate a technology platform purposely designed for out-of-classroom learning 
activities and holistic competencies. This platform is created in a way that is 
student-centred. Its features are based on many pedagogical theories and inno- 
vations and has many features such as rubric templates for competencies, vali- 
dated and reliable instruments for measuring student competencies and various 
prompt feedback options. Using this platform, both direct and indirect evidence 
can be recorded and collected for assessment and other quality assurance re- 
quirements at ease. The platform can also provide an alignment report (in pdf 
format) between programme, course and competency learning outcomes as well 
as assessment as shown in Figure 9.1. The software platform supports web-based 
usage and is available on both Android and iOS devices. The name of the plat- 
form is YOCLE (https://yocle.net). 


Users Assessments Material 


Users Assessments Material Users Assessments Material 


Programme Learning Outcomes (PLOs) 
Course Competency Outcomes (CCOs) 
PLO1: To deliver skills and education 


CCO1: Critical Thinking programmes that enhance engineering 
CCO2: Problem Solving capacity, safety standards, and 
CCO3: Professional Values and Ethics infrastructure that remains safe and fit 


CCO4: Lifelong Learning for purpose, 


EGGS: Geli oreness PLO2: To develop the existing engineering 


MURS Real workforce skills capacity. 

a. BEES PLO3: To bring the engineering community 
CCO7: Teamwork Competency together in order to identify where and 
CCO8: Creativity how the most impactful and sustainable 

interventions. 


Course Learning Outcomes (CLOs) 


Outcomes Alignment 
CLO1: To develop a clear understanding on g 


Course Details workplace and construction safety. 


Title: Oversea Construction CLO2: To learn about sustainable practices in CLOs PLOs 

Safety Course SETZ construction. 
d i 4. + 

ZR CLO3: To connect academic engineers with 

From: 2021-10-12 09:00 Practitioners in the industry and allow 2 v 

To: 2022-10-12 18:00 them to share knowledge and insights. 
Coordinator: Cecilia Chan 3 O 
Covteacharsi( ee Cecilia Chan, Ronke_07 Programme Learning Outcomes oka 2 1 


Assistant(s): 


EE 


D G 


Figure 9.1 YOCLE course showing Programme, course and competency alignment. 
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9.3.1 Case Study — Overseas Construction Safety Course 


For constructions to proceed smoothly and efficiently, safety always plays a 
foundational and significant role. This course is designed to prepare student 
engineers, surveyors and architects to gain a comprehensive understanding of 
workplace and construction safety before they enter the workplace in their desig- 
nated field. Students in this course travel to Southeast Asia and work with indus- 
trial and community partners where they develop and apply safety knowledge, 
skills and competencies on projects in building sites. Students must complete the 
video online course and pass all the quizzes and short questions prior to their 
field visits as shown in Figure 9.2. Most of the projects are located in poorly 
funded areas in Southeast Asia where safety in construction is not a priority. 
With scarce human resources, students are expected to be involved in diverse 
tasks, and will therefore be given plentiful opportunities for hands-on experi- 
ence as they help the community partners deliver projects. Not only will this 
course allow potential students to learn more about safety in the construction 
and engineering contexts, but it will also act as a platform to connect students 
with engineering practitioners in the industry as they are given the opportunities 
to share their experiences and insights. 


Assessment Details 


Title: 
Method: 
From: 

To: 
Weight: 
Participant: 
My Role: 
Status: 


Description 


1. Introductory Quiz 


Introductory Quiz 
MCQ 

2021-10-12 09:00 
2022-10-10 18:00 
1796 

Emma Liu 
Coordinator 

New (^9 months to 
close) 


1. Introductory Quiz 


Before each and everyone of you become eligible 
to work in construction sites, you have to study 
the Draft National Building Code and other 
documents on safety rules. Can you still 
remember those rules now? 


Take this MC test to see how much you can 
remember! 


Material 


6 The Draft National Building Code 


Safety Rules Handbook 


Course Competency Outcome (CCOs) 
Assessment Level 


CCO 1: Critical Thinking 

CCO 2: Problem Solving 

CCO 3: Professional Values and Ethics 
CCO 4: Lifelong Learning 

CCO 5: Self-Awareness 


Standard 
Introductory 
Standard 


° 


© Back 


D 


1. Introductory Quiz 
Assessments 


Which of the following options is not 
a requirement for safety shoes 
used in construction site? 


(10%) 


A. Heat resistance up to 500 °C 
B. Oil resistant sole 

C. Must not have holes or torn 
D. Must have good grip 


ghich of the following options is a 
requirement for high visibility 
coverall used in construction 
site? 


(1096) 


A, Surtan Orange in color 


B. Cotton or mixed Fabric with up to 60% 
cotton 


C. Reflective straps on chest and Q. 


© Back 


D 


Figure 9.2 Screenshots showing video and course materials. Videos and course materials 
can be integrated onto the platform for the course. MCQ quizzes with auto- 
matic correction is available for quick answer checking. 
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The overseas construction safety course lasts three weeks and includes an on- 
line video course with quizzes and short answer questions. Students will need 
to fill in a self-assessed validated evaluation of their holistic competencies at the 
beginning of the course to promote their self-awareness regarding competencies. 
Figure 9.3 shows a student profile, along with their competency scores and a 
sample of the self-assessed evaluation. 

Apart from the self-evaluation, students will also need to complete their learn- 
ing contract as shown in Figure 9.4. Students can fill in the learning contract 
anytime in the system and as you can see, it supports multiple languages. More 
on learning contracts are described in Chapter 4. 

In accordance with the course learning outcomes and the competency out- 
comes, the course encompasses a wide variety of assessment tasks, ranging from 
multiple-choice questions which test participants’ understanding of their course 
content, to reflection which encourages deeper thinking on their learning expe- 
rience and holistic competencies development as shown in Figure 9.5. Through 
different learning and assessment tasks, this course aims at equipping partici- 
pants with a total of eight holistic competencies which are believed to be essen- 
tial for students to excel in their studies and work in the future. 

Students will need to manage their own time in fieldwork, allowing time to 
conduct field research by direct observation and interaction with the environ- 
ment. Direct input on the worksheets via smartphones or tablets makes life really 


YOCLE Survey 


I Questions 


Grace Chan Articulate ideas using verbal means 


Communication 


01 April 
The University of Hong Kong 7 ^ VeryPoor Poor Average 
Hong Kong 

Happy Always La! 


«0l IP 


Single Convey ideas clearly in writing 
Self-Respect O O e Oo 
Activities Global VeryPoor Poor Average ^ Good 
Competency 
Coordinated 1 GS Score 
Lifelong Learning l 
Assisted 1 | Listen attentively 
Eee Professional Values 
‘Assessed (2 25 and Ethics : O O O O 
Very Poor Poor Average Good 
Knowledge 
EU Adaptability 1 Communicate with others effectively using 
adi n N aids 


Q—O0—0—O—O 
Very Poor Poor Average ^ Good Very Good 


Figure 9.3 Student profile and validated self-assessment on holistic competency. 
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Learning Contract ^;:5] $244] 
Nature and purpose of this document £ 3 ATH (fj: 
This is a learning contract between the University and you, a student participant. 
EXER AFAR. 
You represent the university and faculty. 
5. Learning Contract #924) Mele X RAE. 
This document states the expected commitment, conduct and responsibilities of participants to be borne in 
mind at all times and fulfilled during the Safety Construction Programme (hereinafter refer to Programme). 


Description AGGERE T SAT RAR (FRIR WINTER RE. GE, 

Nature and purpose of this document Z:3'(t Assessments DN 

FORERO ELS 1 Please write down two to five competencies that you think that you are particularly "STRONG" at. 

" " e e Sai SUB OR A (RAAB) mie. 

This is a learning contract between the University 2. Please write down two to five competencies that you think that you are particularly "WEAK" at. 

and you, a student participant. Wa 2 RIN AUS RR da" GET RO OJOS E77. 

EASA RZ ln 57 RA. 3. (Think about how you can enhance your competencies): As a participant of the Programme, my 
responsibility is to improve my weak competencies by: 

You represent the university and faculty. GR ote MAIMED) (FARRAR ER Hi et E Im DI, et DR 
VARA TAMETT A com din fie 

AIR RIS IER RS 4. Ihave received, read and understood the information provided in the welcoming bundle. 


RECUA. (dix JEISMOIGD Gub FRE OLI VERE. 


This document states the expected commitment, 5. Ihave agreed to comply with the rules and regulations in the welcoming bundle, including the rules and 


conduct and responsibilities of participants to be regulations of residential halls where | will be staying. 
borne in mind at all times and fulfilled during the dade spo pats. ASA MM. 
Safety Construction Programme (hereinafter refer 6. | agree to avoid racial, religious, sexual and disability discrimination or harassment. 
to Programme). RAGA LM. ad. KAMEN, 
7. By signing this contract, | agree to comply with these guidelines and strive to create a positive learning 
ACERBA T SMBERMLRERE (LA FRAR environment for all of us. 
E) BBARR, TAMRE. WERA. dele fot TAHIR. HERIR HH — 4 EM dm. 


Participants Responsibilities $ Jr # if N íf : 
1. lagreeto enthusiastically participate and contribute in all the activities. 
TARAS SMAAD. 


Course Competency Outcome (CCOs) 2. lagreeto take initiations to improve my competencies. 


Assessment Level DUALS PRT A RCS RAE). 
3. lagree to be honest and open about my expectations, difficulties, emotions and goals to peers and 
CCO 1: Critical Thinking Not assessed mentors. 
CCO 2: Problem Solving TE Heng 8 FOB ski OOM, pe, WEAH. 
à 4. lagreeto seek advice from peers and mentors with an open mind and heart and willingness to learn. 
ccoa Professional else eng Ers RL RIT A RNB 2] R^ de Re CR RR SR LA 
CCO 4: Lifelona Learnina Not assessed 5. lagreeto listen attentively and respect my peers and mentors. 
€ Back RA SAORI ALS AE. 


6. l agree to give feedback and appreciation for support from peers and mentors. 
KAS Fa ed p RH za SRA. 


G 


SIGNATURE DATE 


Figure 9.4 'The YOCLE system showing the learning contract. 


easy for students. Figure 9.6 shows different formats of answer input. A short 
field worksheet is designed to help them summarise their learning experience, 
and grading standards/rubrics are developed and given to students before the 
trip (see Figure 9.7). 

Any assessor can provide feedback. And there are multiple approaches of feed- 
back to choose from, for example, Dictaphone typing, audio and written com- 
ments. General comments, images or pdfs can also be attached via the message 
chatbox and can be sent to the whole class, a group or an individual student. 
Figure 9.8 shows a written feedback and an audio feedback. 

Using technology for assessing experiential learning activities and programmes 
can bring many advantages. One of the benefits is the convenience for teachers 
and community or work supervisors to assess multiple students using the same 
system, hence allowing transparency, collection of evidence, and communica- 
tion. With a system like YOCLE, information about students is automatically 
shared with the community supervisors once the review forms are filled in, and 
they can then be shared through the platform with the teacher and the students 
themselves. It is also more convenient for students and the teacher to access the 
reviews, as they have immediate access to review forms or assessment submis- 
sions. Another advantage is that systems as such allow for feedback in various 
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Information Users 
° 
Assessor 
SS 


2021-10-12: Introductory Quiz 
(MCQ) 


2021-10-12: MCQ on Safety 
(MCQ) 


2021-10-12: Short Questions on 


Safety 
(Reflection) 


2021-10-12: Final Quiz 
(MCQ) 


2021-10-12: Learning Contract 
#5 
(Learning Contract) 


2021-10-12: Reflection - Think 
Think Ponder Ponder 5c HAedg 


NZ) 
(Reflection) 


D 


v ECCHNENINEI 


Oversea Construction Safety Course BIRTZ $i 6. Reflection - Think Think Ponder Ponder SE Oversea Construction Safety Course SM Tei 


Material 


9/0/0 > 


ann > 


900 > 


900 > 


9/0/0 > 


ann > 


1 In the last few days, we believe 
that you should have developed 
the above competencies. Can 
you tell us how you developed 
these competencies? 


(3396) 


The last few days at the safety camp, I learnt a 
lot, how people live together, how different 
people can be, and in fact, how lucky we are. 
There was one time that we went to this 
XXXXXXX 


Uses superficial, simplistic, or 
irrelevant reasoning and 
unjustifiable claims; Makes 
illogical, inconsistent inferences; 
Maintains or defends views based 
on self-interest, regardless of the 
evidence; Ignores significant 
implications, consequences, or 
solutions 


Wee Fair Q 
Follows some evidence to 
yc 
D 


© Back 


a - 


Add Item 


Assessors 


Coordinator 
O ser 
Peers 


Number of peers: 1 


Other assessors 


(C) self-chosen assessors by participants 


o 


Publish 


User names 


e D © 


Back Duplicate Delete 


D 


x 


Peer Assessors 


Select the 1 peer assessors out of the other 9 
participants. 
All peers are assessors 


Davislo 


Emily Fong 


Emma Liu 


Grace Chan 


Jonathan Yeung 


(7 
Lilia Leung eil 
Nellie Wong e 
Participant1 a 


PR sean Dik Hin Colloton 


om Er 


X Clear all 


LI 


Select all 


Figure 9.5 A list of assessment tasks and a sample of the student response in the reflection 
exercise. The teacher has the option to choose different assessors to mark the 
assessment. The options include the teacher, student self, peer students and 
other external assessors. 
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1. Field Trip Assessment 


Attach one photo of a group of rocks and explain why it is 


20% 
sedimentary/metamorphic/igneous. ( e) 


Add description (rock.jpg, 14KB) 


Upload files 


I Upload files i 


Drop them here 
Accepted files (max: 32MB): © 


Figure 9.6 This shows the system allowing flexible submissions in various formats, such 
as pictures, videos, audio and PDF documents. 


CCST9023 The Oceans: Science and Society Field Trip 


Information Assessments Rubri Material 
Poor Fair Good Excellent 
* ** *** keete 
Problem Solving Shows limited | Shows partially Shows clear Shows clear | 
understanding of the | developed | understanding of the | understanding of — | 
problem and broader | understanding of the | problem and identifies | problem and identifies | 
| context At level 1a | problem and | many specific factors | specific factors that 
student should know — | indentifies a few | that influence the influence the approach 


| what the problem asks 
them to find i.e. the 
goal. 


specific factors that | approach to a problem | to a problem before — | 

influence the approach | before solving, At level | solving. At level 4 

to a problem before | 3 a student can clearly | students should be — | 
| 
| 


solving. At level 22 | identify all variables — proficient in all 
| student should | required and separate | previous levels and 
| understand the these from any can reformulate the 

variables and/or | extraneous problem if called upon 

information required | information. to do so. 


and use suitable. 


< ss Hamen ones 


Figure 9.7 This screenshot shows a rubric template. YOCLE consists of over 850 rubrics 
templates for different competencies and they are fully customisable. The sys- 
tem allows teachers to choose from 3 to 6 Likert scales, which can be used to 
assess each skill and produce evidence of student learning. 


forms. In the YOCLE system, supervisors can submit scores according to rubrics 
and also share personalised comments with the students. The support of various 
forms of feedback allows for more insights, and as they can be provided at any 
time and from any location, it is particularly useful for students who may some- 
times feel isolated in a foreign land. During the COVID pandemic, students 
were not able to travel to Hong Kong, and the feedback, messenger and pod- 
casting functions from YOCLE were able to help students to build peer-to-peer 
relationships. 
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6. Reflection - Think Think Ponder Ponder 54: :t$2848 - 818 


E 
Fa really glad that you have developed these competencies. I noticed that you were very confident. 
H 
Cedla Chan 2022-01-22 22:24:25 
3 Is there any particular encounter help you to realise some of your weaknesses? How (34%) 
did you overcome them? Or how will you overcome them in the future? 
š ° 
<= EES 
Figure 9.8 Assessor feedback via multiple approaches e.g. Dictaphone typing, audio and 
written. 
Conclusions 


One of the biggest challenges in relation to technology use in experiential learn- 
ing is students’ and teachers’ lack of technical competence to cope with the new 
tools. Students often need technical training in technology use and clear instruc- 
tions from teachers in order to enjoy the benefits of using current tools in learning 
(Murphrey, 2010). It means that students have to put extra effort into learning 
new skills (Le et al., 2015), and this may add to the academic burden they al- 
ready face. Teachers, on the other hand, need to acquire special skills that are 
necessary for utilising technological tools and creating virtual learning scenarios 
(Jarvis & Dickie, 2010; Le et al., 2015). Moreover, the creation of tthe use of 
some technological tools incurs high costs (Angolia & Pagliari, 2018; Beckem & 
Watkins, 2012). Before a technological tool is integrated into teaching and learn- 
ing, teachers often need to do some preparation in advance, for example, setting 
up accounts, creating virtual groups, familiarising themselves with the tool, and 
giving students an orientation to the tool/programme (Leonard et al., 2011). 
Such preparation work can cause an increase in teachers’ workload. 

As technologies are fast evolving, it is challenging for teachers to keep 
abreast of the latest developments in the field (Mathews & Flynn, 2018). In 
lesson planning, the type of technological tools and their versions need to be 
carefully selected to avoid the risk of obsolescence and system incompatibility 
(Sandlin, 2014). Effective integration of technology necessitates prior pilot- 
ing of tools and planning for alternative methods by the teacher. As Rawlins 
and Kehrwald (2014) put it, “there is a need to reconceptualise approaches 
to create learning environments which make the best use of the available 
technologies” (p. 215). 

Despite its potential to facilitate learning, technology can segregate students 
who have to work with a slow Internet connection or those who have limited 
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access to technology. In synchronous activities involving students from different 
locations, those in remote locations may not be able to participate in some activi- 
ties, and thus feel excluded from the learning experience (Mundkur & Ellickson, 
2012). Therefore, whether technology should be used and the extent to which it 
should be integrated into experiential learning are decisions that require careful 
consideration. 

The question of whether technology enhances learning and to what extent it does 
so remain a topic of ongoing debate. Most of the studies that assessed technolo- 
gy-enhanced experiential learning tend to depend on students’ self-reported expe- 
riences, with very few exploiting course examinations and objective tests for the 
measurement of learning attainment. While positive learning experiences have been 
widely reported (e.g., Lowell & Alshammari, 2019; Marriott et al., 2015), empirical 
studies that measured students’ learning attainment have yielded mixed results. Such 
inconclusive findings indicate the need for more rigorous empirical research into the 
assessment of technology-enhanced experiential learning, and the impact of technol- 
ogy on experiential learning. 


Questions to Ponder 


e Why are you using technology for the experiential learning assessment? 
e Does technology actually help deliver the expected outcomes? How? 
e Are you burdening the students and the partners by using technology? 
e Are you able to support diversity, equity and inclusion? 


Personal Reflection 


We, humans, can be so stubborn; we don’t like changes. Unless there is a 
world-shattering situation that drives us to it. The COVID-19 pandemic is 
one example, I guess. We have been trying to encourage teachers to adopt 
technology in their curriculum for over a decade, but it is not until the 
pandemic that teachers have had to change, and they have begun to adapt. 
But once they started to adapt, they are back to that stubborn state again. 
So now that most teachers use one particular video platform, for example, 
Zoom, introducing them to another video conferencing platform would 
be almost impossible. 

That is why it is important to choose your technology wisely. Once a 
decision is made, we can be too stubborn to change it. Always consider 
your pedagogy before your technology. 
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10 Quality Assurance 
and Evaluation in 
Experiential Learning 


An oversight: Some may believe a large sample of quantitative data is a better 
representation for evaluation but sometimes a deep meaningful case study may 
be a better indicator to inspire. 

— Chan, CKY 


Introduction 


Experiential learning programmes and activities do not traditionally follow the 
usual learning path in a degree programme, and often require more time and 
funding resources. Thus, teachers need to provide rationales and justifications 
for why this learning pedagogy is effective and efficient, and worthy of contin- 
ued support. 

Monitoring and evaluation of experiential learning programmes are vital, and 
they provide funders, management, teachers and students with information on 
the programme’s rationales, relevant recommendations and quality assurance. 
These evaluation processes help determine whether the programme works for 
the target student group, refine programme delivery, identify any ongoing con- 
cerns, and provide evidence and justification for the continuing support of the 
programme. 


10.1 Evaluation Questions 


Evaluation questions, similar to research questions in academic research projects, 
provide the purposes, and in turn, guide the methods used to collect data to 
understand the problem under investigation. There are four categories of evalua- 
tion questions for experiential learning programmes, namely: Needs evaluation; 
Process evaluation; Impact and outcome evaluation and Economic evaluation. 


Needs Evaluation 


Needs evaluation — this identifies the targeted student group for the experiential 
learning programme, the rationale for such programme and the actual needs 
fulfilled by the programme for this group of students. 
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Sample questions related to this category 


e What are the characteristics of this targeted student group? 
e What are their needs? 
e Why is experiential learning programme suitable for this target group? 


Process Evaluation 


Process evaluation - this examines whether the experiential learning programme 
was implemented according to plan and whether the target student population 
was reached. 


Sample questions related to this category 


e Were your programme’s activities put into place as originally intended? 

° Did the programme address the challenge or the needs of the programme? 
How? 

e Was the target student population adequately reached and involved in the 
activities? 

e How did the target population interact with the programme? 

e How was the programme functioning from administrative, organizational, 
and teacher perspectives? 


Impact and Outcome Evaluation 


Impact and outcome evaluation — this determines whether the programme has 
brought about a change and successfully achieved the expected outcomes. 


Sample questions related to this category 


e Have the programme’s outcomes, objectives, and goals been achieved? 

° Is the programme beneficial to the target student population? 

° Did the programme produce or contribute to the intended outcomes in the 
short, medium and long term? How? 

e What unintended outcomes (positive and negative) were produced? 


Economic Evaluation 


Economic evaluation — this evaluates the cost effectiveness of the programme 


Sample questions related to this category 


e What has been the ratio of costs to benefits? 

° Does the value or benefit of achieving the programme's goals and objectives 
exceed the cost of producing them? This may include time, staff and other 
non-monetary costs. 

e What is the most cost-effective option? 

e Are there alternative pedagogical approaches that could achieve the same 
outcomes at a lower cost? 
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Depending on the reason why evaluation is needed, for instance, intentions such as 
documenting programme progress, demonstrating accountability to the different 
stakeholders, or identifying ways to improve the student learning experience, the 
evaluation purposes and questions will guide the choice of methods used. 


10.2 Evaluation Methods 


The methods employed depend on the evaluation purposes and questions, and 
the common methods are quantitative, qualitative and mixed methods. Quan- 
titative methods involve the collection of numerical data, which can be used 
to investigate the who and what of the experiential learning programme, such 
as the number of people who participated and the number of participants who 
completed the programme. Quantitative data can be collected via surveys, pre- 
tests and post-tests surveys, observations, or reviews of existing assessment, doc- 
uments and databases. 

Qualitative methods involve a collection of qualitative data which can be used 
to gain a more in-depth understanding of the processes, behaviours, feelings, 
actions, environments and relationships of the social and cultural aspect of the 
programme, i.e., the why question. Unlike quantitative evaluation methods 
which can test clear-cut hypotheses, qualitative evaluation methods can reveal 
unexpected outcomes through observations, case studies, face-to-face individ- 
ual or focus group interviews, and projective techniques such as photolanguage 
(Bessell et al., 2007). Photolanguage makes use of photos or pictures to help 
participants to stimulate communications. 


10.3 Literature Review on Evaluation of Experiential 
Learning Programmes 


As summarized by Gosenpud (1990), evaluation of experiential learning can 
be carried out either by comparing experiential learning with other pedagogi- 
cal methods on certain criterion variables, or comparing changes regarding the 
criterion before and after the implementation of an experiential learning pro- 
gramme. This latter falls under a category referred to as “straight evaluation 
study.” Using straight evaluation study, scholars have conducted evaluations on 
the influence of experiential learning on cognitive learning, attitudes and behav- 
ioural change/skill acquisition (Gosenpud, 1990). Depending on the purpose of 
evaluation, different criteria can be used to evaluate experiential learning pro- 
grammes. Ways to measure three common criteria are listed below: 


° Cognitive learning: comparison of exam grades/ objective tests; 

e Attitude change (attitude towards course/ enjoyment/ feelings of accom- 
plishment/ benefits of class): questionnaire; 

e Behavioural change/skill acquisition: open-ended questions to co-workers 
or supervisors/self-reports/examination on acquired skill/co-worker rating 
or observation. 
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It should be noted that studies have supported the use of both qualitative and 
quantitative methods for evaluation, as the two types of methods can complement 
each other, and when used together, strengthen the evaluation process (McLeod & 
Craig, 2004). It is also common for studies evaluating the effectiveness of ex- 
periential learning programmes to adopt mixed methods, i.e., both quantitative 
and qualitative measures, or for surveys to have open-ended questions to collect 
qualitative data on top of quantitative items (Hanneman, 2013; Mak et al., 2017; 
McLeod & Craig, 2004; Weinberg et al., 2011). 


10.3.1 Surveys 


As reflected by a series of recent studies on the impact of experiential learn- 
ing, surveys or questionnaires are one of the most common instruments used 
for evaluating experiential learning programmes (Hanneman, 2013; Kruger 
et al., 2015; Mak et al., 2017; McLeod & Craig, 2004). Each study ad ministers 
questionnaires differently. For some studies, both pre-test and post-test are con- 
ducted to obtain comprehensive data over time, while some only conduct post- 
test due to various reasons and constraints. 

The evaluation conducted by Hanneman (2013) analyses the effectiveness of 
an experiential environmental programme in the USA in fostering long-term 
personal responsibility and awareness about ocean pollution. The investigator 
developed a survey instrument for the research that consisted of four sections: 
a sociodemographic and field trip experience section, a question and drawing 
section which inquires students about their concepts of ocean pollution, five 
Likert-scale response questions, and two open-ended questions. As this study is 
intended to be a long-term study, the evaluation was conducted with participants 
of the programme for five years. 

Savage and colleagues (2015) utilised a multipart self-assessment survey to exam- 
ine 32 students’ perceptions of their confidence and competence before and after 
a sustainability leadership certificate programme. A pre-survey and a post-survey 
were conducted, which contained 15 questions each that required the participants 
to rate their confidence and competence levels on a scale of one (disagree) to four 
(agree). The post-survey also included qualitative questions for the participants to 
explain the increase, decrease, or lack of change in their scores. The results show 
an increase in the participants’ perceived confidence and competence. In addition, 
from the analysis of the qualitative data, the participants appeared to be able to 
describe their learning outcomes in line with the programme design. 

In McElwain et al.’s (2016) study, a pre-test and post-test to evaluate pro- 
gramme effectiveness were administered to the service recipients, instead of the 
undergraduate students who carried out the service-learning project. A total of 
1,005 adolescents who received the relationship education sessions taught by the 
college student facilitators responded to the pre-test and post-test comprising 
of three components: demographic questions; knowledge of social science and 
items that were designed to measure depressive symptoms, individual empow- 
erment, conflict management, attitudes towards marriage, faulty relationship 
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beliefs, and dating aggression. Although the study yielded positive findings, the 
researchers argued that an analysis of the undergraduates’ reflection journals 
would have provided better insights into the effectiveness of the programme. 
In addition, continuous feedback from the undergraduates and the supervising 
faculty members was said to be gathered for programme improvement, but the 
instruments and methods used were not explained and reported in detail. 

Apart from conventional paper-form surveys, researchers also utilize online 
platforms to facilitate their studies. The study conducted by Kruger, Kruger and 
Suzuki in 2015 made use of online surveys to evaluate the effectiveness of ex- 
periential learning in a free clinic run by students majoring in Physical Therapy, 
Health Education or Health Administration. The survey was conducted online 
via Qualtrics, and the URL of the survey was distributed through mass email to 
all volunteers at the free clinic. The survey was also distributed through Face- 
book posts on the free clinic Facebook page, as well as individual emails to the 
Board of Directors of the free clinic (Kruger et al., 2015). The survey consisted 
of 18 five-point Liker-scale statements belonging to four categories: Preparation 
and Fit, Quality of Volunteer Experience, Skill Development, and Community 
Impact. The survey examines students’ self-reported experiences volunteering at 
the free clinic in the above four aspects. 

Ickes and McMullen (2016) conducted an online survey of student health 
coaches’ self-perceived knowledge, skills, and self-efficacy using a pre-test/post- 
test design. All the student health coaches in a health behaviour course were 
invited to participate in the study by e-mail, and the pre- and post-surveys were 
administered through an online survey software, Qualtrics. The pre-survey and 
post-survey contained 33 and 36 questions respectively, all aimed at measuring 
students’ gain in knowledge and self-efficacy as reflected by their self-ratings 
on a four-point Likert-type scale. Improvements in the scores show that the 
experience-based health behaviour course was effective for developing the neces- 
sary knowledge, skills, and self-efficacy for a health coach. 


10.3.2 Objective Assessment Results 


Examination grades can be a direct indicator of student performance, especially cog- 
nitive and academic performances. By comparing examination grades or results of 
certain objective tasks, it is possible for researchers to know whether experiential 
learning is effective in terms of enhancing cognitive development. Nonetheless, it 
should be noted that grades as measures of academic achievement are challenged for 
their questionable validity and objectivity. In other words, academic grades do not 
always reflect students’ level of academic achievement accurately (Allen, 2005). In 
the case of experiential learning, teachers’ grading of assessments such as the reflec- 
tive journals can be highly subjective and, to a certain extent, biased (Brail, 2016). 
This poses even more problems on the use of exam grades as criterion variables, for 
example, ifa student obtains a higher grade after completing an experiential learning 
programme, it is possible that the higher grade comes from a biased judgement of 
the supervisor, instead of an improved academic performance. 
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10.3.3 Peer/Supervisor Observations and Ratings 


Observation or rating by peers or supervisors are the most common ways to doc- 
ument behavioural changes or the acquisition of new skills. This mode of eval- 
uation is especially popular in clinical placements in which supervisors observe 
performances of medical trainees as a component of assessment (Kogan et al., 
2009). During clinical placements, medical trainees have the opportunity to 
interact with actual patients. Through observing and assessing medical trainees 
with patients and providing feedback, instructors can effectively help students 
acquire new skills and improve on their weaknesses (Kogan et al., 2009). Apart 
from clinical placements, observation and rating are also adopted in the HAVE, 
U Can Programme (programme mentioned in Chapter 4), which aims to en- 
hance holistic competencies and virtues, including consideration, leadership, 
manners, resilience, self-confidence and teamwork. Through daily observations, 
mentors grade student participants according to the above aspects and document 
whether improvements have occurred. Mentors’ close and trusting relationships 
with the mentees and prompt feedback via observation facilitated behavioural 
changes in students. 

To evaluate the pedagogical outcomes of a client-based capstone course in a 
healthcare administration programme, Dominguez et al. (2009) reported the 
use of a survey completed by course professors at the end of the course. The 
survey contained items on 12 competencies such as communication, change 
management, collaboration, and professionalism. Course professors rated their 
students’ abilities in the 12 areas on a nine-point Likert-type scale ranging from 
one (well-below expectations) to nine (well above expectations). 


10.3.4 Multi-method Approaches 


Some studies have reported the use of a combination of quantitative and quali- 
tative methods to triangulate data from multiple sources in order to evaluate the 
effectiveness of experiential learning programmes. Kassabgy and El-Din (2013) 
investigated the impact of a service-learning course component on students’ ac- 
ademic enhancement, civic engagement, personal growth as well as the attitudes 
and perceptions of the campus community. In the service-learning project, the 
students provided English language tutoring to the custodians, security guards, 
and housekeepers who worked for the university. The impact of the project was 
evaluated through an analysis of the students’ reflection papers, their responses 
to an online questionnaire, as well as the data from focus group interviews with 
the students and university workers respectively. The findings show that the 
service-learning project helped the students achieve the intended learning out- 
comes of the course, enhanced their civic responsibility, and promoted positive 
attitudes and personal skills such as tolerance, self-confidence, and communi- 
cation skills. As a result of participating in the project, the university workers 
reported a desire to continue learning English and that they would recommend 
the course to other university workers. 
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In another study, Boneck et al. (2014) examined the effects of service- 
learning projects within the Voluntary Income Tax Assistance (VITA) pro- 
gramme in accounting education based on the data collected from the students 
and tax clients who participated in the programme. The students were asked 
to rate statements about their competencies before and after the programme 
on a scale of one (strongly disagree) to seven (strongly agree) in four areas: 
interview effectiveness, intake form proficiency, software proficiency, and com- 
pletion competence. In addition, the students were asked to describe their 
perceptions of the service-learning programme before and after the activity. 
The third set of data was the tax clients’ participation experiences in the pro- 
gramme, gathered from their written statements in an exit survey. The analysis 
of the quantitative and qualitative data shows that the VITA service-learning 
experience helped students develop professional skills in accounting and a pos- 
itive attitude towards community service, at the same time providing quality 
tax services to the community. 

Rodriguez (2018) employed a mixed-method approach to collect data from 
student participants, teachers, and other participants in 15 projects in architec- 
tural education. The data analysed included students’ individual and focus group 
feedback; peer feedback; feedback from the community partners, users, and ad- 
visors; teacher observations; and materials produced by the students such as log- 
books, videos, photographs, and drawings. Based on the data from multiple 
sources, she found that the projects provided opportunities for the students to 
work on real-life design problems, enabled them to achieve the intended learning 
outcomes, and increased their intrinsic and extrinsic motivation. 

To evaluate the success of a programme introducing quality improvement in 
nursing education, Kyrkjebo et al. (2001) implemented a two-part questionnaire 
that a group of nursing students completed before and after a two-day theo- 
retical introductory course on the topic to measure their understanding of the 
concepts. In addition, the group reports submitted by students at the end of 
the experiential programme were analysed to examine how they had applied 
the quality improvement tools and techniques in specific situations. The results 
from the quantitative and qualitative analyses revealed that the intervention pro- 
gramme could raise students’ awareness of quality improvement and provide 
them with the necessary skills and knowledge to integrate quality improvement 
in their future professional practice. 


10.4 International Holistic Competency Foundation 
(IHCF) Quality Assurance for Holistic Competency 
Outcomes — Upcoming Future Standards 


Throughout the book, I have repeatedly emphasised the holistic competency 
outcomes that can be developed in experiential learning. Experiential learning 
has become the solution in higher education in recent years to promote the 
development of holistic competencies with the aim of enhancing graduates’ 
employability. This is of course, very difficult to evaluate objectively and with 
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clear evidence of student learning. As part of my research and practice, I have 
been working with a large number of universities, communities and industrial 
partners, and have set up the International Holistic Competency Foundation 
(https: //www.ihcfoundation.net/) 

The International Holistic Competency Foundation (IHCF) is an initiative 
to promote the recognition of holistic competency development in programmes 
through formal certification. There are 12 competency outcomes covered by the 
foundation’s certification system: communication, creativity, critical thinking, 
global competency, information literacy, leadership, lifelong learning, problem 
solving, professional values and ethics, resilience, self-awareness, and teamwork 
competency. Figure 10.1 shows the 12 competencies that are currently being 
certified. 

Courses that wish to join the certification scheme have to satisfy four criteria 
(see Figure 10.2 for the process): 


— 


The course must have clear competency outcomes; 

2 The competency outcome(s) must be one or more of the twelve competen- 
cies identified by the International Holistic Competency Research Partners; 

3 Appropriate assessment approaches (formative and/or summative) must be 
employed to provide evidence on each competency outcome; and 

4 All courses certified will use the holistic competency validated instruments 

to help students reflect on their holistic competency and enhance their 

self-awareness. 


Courses applying for the holistic competency certification will be reviewed by the 
credential committee consisting of educational experts, industry leaders, and com- 


munity partners. Upon approval, the course will receive an internationally recog- 
nised certificate that is valid for two years, after which quality assurance procedures 


000000 


Communication Creativity Critical Thinking Global Competency Information Literacy Leadership 
(CCO1) (CCO2) (CCO3) (CCO4) (CCO5) (CCO6) 


Problem Solving Professional Values Resilience Self-Awareness Teamwork 
(CCO8) and Ethics (CCO10) (CCO11) Competency (CCO12) 
(CCO9) 


Lifelong Learning 
(CCO7) 


Figure 10.1 The 12 holistic competency certified badges. 
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A » E 


Online Application for Course Review and Fi Approval by 
Certification by Course Coordinators Credential Team Credential Committee 


Students Receive Transcripts* Certified Courses ficial Cri ia ued with 
Upon Completion of the Courses Offered to Students Quality Assurance (Every 2 years) 


Figure 10.2 The certification process of the IHCF. 


will be carried out. Students taking the course will be given a transcript summaris- 
ing their achievements in each certified holistic competency as shown in Figure 10.3. 

The International Holistic Competency Certification can serve as a useful tool 
for quality assurance of experiential learning programmes. There is an increasing 
demand for quality assurance and accountability in higher education. As a result, 
colleges and universities have to demonstrate to students, parents, employers, ac- 
creditation agencies, and any interested stakeholders that their programmes meet 
the needs of society and that their outcomes are aligned with industry expecta- 
tions and national standards for quality education. One of the indicators of quality 
education is the employability of their graduates, or in other words, the extent to 
which their students can demonstrate holistic competencies as job-ready gradu- 
ates. In light of this, formal certification of competency attainment serves as port- 
able evidence that students can present to potential employers about the attributes 
they possess and their ability to succeed in the workplace. Chan and Luk’s (2021) 
study involving 2,150 undergraduates and 215 academics from six Hong Kong 
universities shows that both academics and students are in favour of formal assess- 
ment and record of holistic competency development. In another study, Chan and 
Yeung (2021) found that students perceived formal assessment of holistic compe- 
tencies as appropriate in general. Despite the consensus among researchers and 
practitioners concerning the importance of assessing holistic competencies, there 
is very little research on how holistic competencies are certified, and the current 
process of documenting, reporting, and certifying holistic competencies in higher 
education appears to be less than satisfactory (Chan & Chen, 2021). 

The International Holistic Competency Certification is an effort to advance 
the recognition of holistic competencies in education. It also provides a means 
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Figure 10.3 A sample competency development report. 


for ensuring that the assessment of holistic competencies is aligned with in- 
tended programme outcomes and that student achievements are properly trans- 
lated, documented, and recognised in the form of an official transcript. Since 
experiential learning is increasingly adopted by institutions of higher learning 
to promote holistic competency development, the International Holistic Com- 
petency Certification Scheme offers a useful tool for quality assurance of expe- 
riential learning programmes by ensuring that certified programmes have clear 
competency outcomes, utilise appropriate assessment approaches, and document 
competency outcomes correctly. As highlighted in Chan and Chen’s (2021) sys- 
tematic review on holistic competency recognition, there is an urgent need for 
standardised reporting of holistic competency attainment and a shared discourse 
between higher education and industry. 


Conclusions 


This final chapter concludes by providing various evaluation methods and a review 
of the type of evaluation that has been commonly used in experiential learning. 
I also introduce the International Holistic Competency Foundation and its qual- 
ity assurance process for holistic competency which I believe will be an important 
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requirement for experiential learning programmes and extra-curricular activities 
in the near future. The International Holistic Competency Foundation aims to 
enhance university-industry-community engagement by making explicit the com- 
petencies assessed in educational institutions, clarifying what those competencies 
mean, and incorporating the views of industry and community experts in the certi- 
fication process. Hence, the certification scheme ensures that certified programmes 
meet industry and public expectations and that students who have undergone the 
programmes are well-prepared for the employment market and society. 


Questions to Ponder 


e What is a world without quality? 

e What is a world with too much quality control? 

e How do you ensure you do not focus too much on quality assurance 
and lost sight of the meaning, fun and challenge embraced in experi- 
ential learning? 

e Your university only puts emphasis on quantitative data, particularly 
on the number of students registered, but it is difficult to manage a 
large number of participants in experiential learning activities, what 
should you do? 

e It feels wrong to ask the students for their story for the sake of quality 
assurance, how should that be done? 


Personal Reflection 


What makes experiential learning powerful is often the involvement of 
the community. This authentic experience gives rise to many humanistic 
factors that students have to encounter. Students invest personal emotions 
into dealing with the social, cultural and economic challenges and sensi- 
tivities that occur, and these are the true jewels of experiential learning. 

So, for my last reflection, I want to remind generous funders, systematic 
administrators and strict policymakers who may pick up this book. I have 
been trying to provide multiple perspectives from different stakeholders 
throughout the book as it is important to see from others’ point of view. 
Experiential learning assessment and evaluation may be unconventional, 
the quality assurance and control processes may be messy and cloudy, but 
it is exactly because of the authentic, ill-defined, unknown nature of expe- 
riential learning that helps our students to develop creativity, teamwork, 
empathy and many more in this unforeseen future. Do not give in to me- 
diocrity because of something we are not familiar with. 
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The Final Conclusion 


This concludes the ten chapters in assessing experiential learning. I enjoyed the 
process of writing this book, particularly the last months of it. 

There are a few chapters that I find most interesting, as I researched and learnt 
more and more about the nuances. I particularly enjoyed Chapter 5 on reflec- 
tion, as I see how much we depend on it, yet how little we know about it. There 
is more work to be done, and like that drawing by Sean Colloton portraying 
Hamlet's “To be or not to be, that is the question,” I see myself having the same 
struggle when it comes to reflection — “to assess or not to assess.” Efforts are 
needed to improve reflection literacy for both teachers and students. 

I also enjoyed researching for and writing Chapter 7 on ethics. The cases that 
I developed were so useful and eye opening for my team and myself that I am 
already planning my series of professional development workshops around this 
topic. I like the drawing ofthe teacher portraying the Lady Justice, and as teach- 
ers, we need to role model this act of ethics. Research on this topic is still very 
scarce, and this will be an area of research that I anticipate more as upcoming 
development in the next few years. 

Feedback in Chapter 6 is an important topic in its own right. I emphasise 
a lot on the relationship of the assessors and the socio-emotional aspects of 
it, as I think this aspect of feedback is particularly important for experiential 
learning. Of course, if students can develop evaluative judgement ability to 
assess themselves in holistic competency, the above issue will lessen. Thus, I 
anticipate the research study and instrument being developed by my student, 
Jiahui Luo, who specialising in this area will be of significant importance in 
the future to come. 

I am also particularly glad that I introduced the International Holistic Com- 
petency Certification in Chapter 10 for quality assurance purposes, as this is the 
beginning of an imperative era in experiential learning programmes and holistic 
competency. Holistic competency is far too important to ignore, as we need to 
shift the mindsets of teachers so that our students will be ready for the future 
and be able to embrace the rapidly changing society. 

With the ongoing challenge of the coronavirus disease pandemic, more dif- 
ferent innovative teaching and learning activities have emerged and will con- 
tinue to do so, and we are beginning to see the emergence of new assessment 
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approaches and technological tools in this area. Massive Open Online Courses 
(MOOCs) and other online education courses have evolved with the advances in 
technology. These courses are catalysed by the pandemic when previously, learn- 
ers’ mentality and perception towards online education were still infantile and 
full of distrust. Online education set up by international industries is entering 
the post-secondary education market as an influential force in order to possess 
a market share of the neoliberal education industry. International corporations 
such as Google and Alibaba believe university degrees may be a distant past 
(Bariso, 2020) and that they can readily prepare a workforce for their indus- 
tries in a matter of months by providing intensive courses. Some multi-national 
industries such as Apple, Ernst and Young (UK), and IBM have dropped the 
requirement of a degree as a criterion for graduate recruitment as they found 
“no evidence that success at university is linked to achievement in professional 
assessments” (Havergal, 2015). There seems to be an oversight that the purpose 
of post-secondary education is simply to prepare someone for a specific job — the 
end outcome; while other purposes of the post-secondary education are forgot- 
ten, particularly the learning journey — the process itself. Maybe universities have 
indirectly lost sight of this learning process, too, with constant external scrutiny 
and traditional mindsets and approaches assuming that students are only there 
to gain disciplinary knowledge. We need to ensure that a student’s learning jour- 
ney is planned at the forefront. Post-secondary education provides students the 
opportunities to build their characters, their holistic competencies, their resil- 
ience, values and morals, and these are all equally important for the society. We 
need to think of post-secondary education beyond mere job preparation, and 
we need to help students, their parents, and the public see beyond that as well. 

In the future, universities MUST work collaboratively with the community 
on research and teaching. Curriculum needs to be designed with partners in the 
community to allow students to experience meaningful, ill-defined, authentic 
projects and environments, which will add value to the students’ learning ex- 
perience and at the same time, reestablish the purposes of university education. 
Thus, experiential learning will continue to have a considerable role to play in 
education. 

Finally, I want to reiterate the quote used in Chapter 4: 


Assessment in the future may not be the kind of assessment that you and I 
traditional grew up with. The word assessment may mean something com- 
pletely different. Or new term may need to coin for the type of assessment 
deems suitable for experiential learning. 

(Chan, CKY) 


Whether the term — assessment (or any new term) is used to grade, evaluate, 
provide feedback or maybe to badge in the future, it requires us to be literate 
and insightful in experiential learning assessment. Assessment of experiential 
education is only in its infancy. This area needs to grow and develop, and more 
research and scholarships for teaching and learning will make it possible. 
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As the way forward, I foresee new research studies and teaching practices in 
the assessment of experiential learning will transpire. I hope that I have provided 
teachers and researchers food for thought in the various areas of experiential 
learning. 

Thank you for taking your time to read it. 
Yours sincerely, 
Cecilia Ka Yuk Chan 
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